Stream laundry list generation #16

Open
opened 2025-12-29 02:20:35 +00:00 by scossu · 0 comments
Owner

LL generation could take large amounts of RAM for a large submission, as it has to keep the whole CSV in memory. If the CSV output can be streamed, the memory usage would be much more contained and predictable.

The CSV library currently used, ftcsv, does not allow to output the CSV headers separately from the data body, which is required for this to work. Some experiments on a ftcsv fork have been made to implement a separate head/body output.

LL generation could take large amounts of RAM for a large submission, as it has to keep the whole CSV in memory. If the CSV output can be streamed, the memory usage would be much more contained and predictable. The CSV library currently used, ftcsv, does not allow to output the CSV headers separately from the data body, which is required for this to work. Some experiments on a ftcsv fork have been made to implement a separate head/body output.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
scossu/pocket_archive#16
No description provided.