Use external MD for collection long description.

.gitmodules

@@ -1,3 +1,6 @@
 [submodule "ext/darkhttpd"]
 	path = ext/darkhttpd
 	url = https://github.com/emikulic/darkhttpd.git
+[submodule "ext/markdown"]
+	path = ext/markdown
+	url = https://github.com/mpeterv/markdown.git

README.md

@@ -225,88 +225,4 @@ foundational library, Volksdata, has been developed as a spare-time project for
 
 ### Road map
 
-The first goal is to build a working prototype, with all the basic functional
-components, even if not entirely developed or only usable in a specific
-development environment, to demonstrate the overall workflows and
-functionality.
-
-The second step is to produce a basic application, which is fully
-functional and available for use by the intended audience.
-
-#### Prototype (alpha)
-
-❏ = pending; ⚒ = in progress; ⎊ = blocked; ✓ = complete; ✖︎ = not implemented.
-
-- ✓ Configuration + config parser
-  - ✓ Application
-  - ✓ Content model
-    - ✓ Validation rules
-- ✓ Submission module
-  - ✓ SIP building
-  - ✓ Metadata from LL
-  - ✓ Brick structures
-  - ✓ Structure inference
-- ✓ HTML generator
-  - ✓ Index
-  - ✓ Resource
-  - ✓ Static assets
-- ✓ Non-HTML generators
-  - ✓ RDF (turtle)
-  - ✓ Transformers
-  - ✓ JS search engine index
-- ✓ CLI
-  - ✓ Init archive
-  - ✓ Deposit
-  - ✓ Generate site
-  - ✓ Generate LL (single resource)
-  - ✓ Generate RDF (single resource)
-- ✓ Front end
-  - ✓ JS search engine
-  - ✓ Add collections to index page
-  - ✓ Basic styling
-      - ✓ Default type icons
-- ✓ QA
-  - ✓ ~50 resource data set
-
-#### Basic application (beta -> v1.0)
-
-- ✖︎ Management UI & API
-  - ✖︎ Deposit via single tar or zip file submission
-- ⚒ Deposit via local hot folder
-- ❏ Proper collection handling
-  - ❏ Dedicated template
-  - ❏ Link to markdown doc for presentation page
-  - ❏ Handle artifacts as members
-- ❏ Content model
-  - ❏ Multilingual support
-  - ❏ Local overrides
-  - ❏ Relatioships inference
-  - ❏ Markdown support for individual fields
-- ❏ Generator
-  - ❏ htmlgen option for local file or webserver URL generation
-  - ❏ Generate LL (multi)
-  - ❏ Generate RDF (multi)
-- ❏ Front end
-  - ❏ Category browsing
-  - ❏ Improve search indexing
-  - ❏ Enhanced styling and access
-- ❏ CLI
-  - ❏ Dump & restore (whole archive & individual resources)
-- ❏ Testing
-    - ❏ Unit tests
-    - ❏ >100 resource data set
-- ❏ Documentation
-  - ❏ Break main sections off README
-  - ❏ Submission guide
-  - ❏ Site generation guide
-  - ❏ Content modeling guide (including proxy concepts)
-
-#### Post-basic
-
-- Deposit via remote hot folder
-  - FTP
-  - S3
-- Incremental build
-- Rebuild only site assets
-- Custom templating
-- Deposit via S3 source pointer
+See [Road map doc](doc/roadmap.md).

config/model/typedef/collection.lua

@@ -15,6 +15,15 @@ return {
             type = "resource",
             range = {file = true},
         },
+        long_description = {
+            uri = "pas:longDescription",
+            label = "Long description",
+            description =
+                [[Text document in Markdown format with a detailed
+                description of the collection.]],
+            type = "resource",
+            range = {file = true},
+        },
     }
 }
 

doc/roadmap.md

@@ -0,0 +1,89 @@
+# Pocket Archive Road Map
+
+The first goal is to build a working prototype, with all the basic functional
+components, even if not entirely developed or only usable in a specific
+development environment, to demonstrate the overall workflows and
+functionality.
+
+The second step is to produce a basic application, which is fully
+functional and available for use by the intended audience.
+
+## Prototype (alpha)
+
+❏ = pending; ⚒ = in progress; ⎊ = blocked; ✓ = complete; ✖︎ = not implemented.
+
+- ✓ Configuration + config parser
+  - ✓ Application
+  - ✓ Content model
+    - ✓ Validation rules
+- ✓ Submission module
+  - ✓ SIP building
+  - ✓ Metadata from LL
+  - ✓ Brick structures
+  - ✓ Structure inference
+- ✓ HTML generator
+  - ✓ Index
+  - ✓ Resource
+  - ✓ Static assets
+- ✓ Non-HTML generators
+  - ✓ RDF (turtle)
+  - ✓ Transformers
+  - ✓ JS search engine index
+- ✓ CLI
+  - ✓ Init archive
+  - ✓ Deposit
+  - ✓ Generate site
+  - ✓ Generate LL (single resource)
+  - ✓ Generate RDF (single resource)
+- ✓ Front end
+  - ✓ JS search engine
+  - ✓ Add collections to index page
+  - ✓ Basic styling
+      - ✓ Default type icons
+- ✓ QA
+  - ✓ ~50 resource data set
+
+## Basic application (beta -> v1.0)
+
+- ✖︎ Management UI & API
+  - ✖︎ Deposit via single tar or zip file submission
+- ⚒ Deposit via local hot folder
+  - ❏ Option to clean up sources & LL on success
+- ⚒ Proper collection handling
+  - ✓ Dedicated template
+  - ✓ Link to markdown doc for presentation page
+  - ❏ Handle artifacts as members
+- ❏ Content model
+  - ❏ Multilingual support
+  - ❏ Local overrides
+  - ❏ Relatioships inference
+  - ❏ Markdown support for individual fields
+- ❏ Generator
+  - ❏ htmlgen option for local file or webserver URL generation
+  - ❏ Generate LL (multi)
+  - ❏ Generate RDF (multi)
+- ❏ Front end
+  - ❏ Category browsing
+  - ❏ Improve search indexing
+  - ❏ Enhanced styling and access
+- ❏ CLI
+  - ❏ Dump & restore (whole archive & individual resources)
+- ❏ Testing
+    - ❏ Unit tests
+    - ❏ >100 resource data set
+- ❏ Documentation
+  - ❏ Break main sections off README
+  - ❏ Submission guide
+  - ❏ Site generation guide
+  - ❏ Content modeling guide (including proxy concepts)
+
+## Post-basic
+
+- Deposit via remote hot folder
+  - FTP
+  - S3
+- Incremental build
+- Rebuild only site assets
+- Custom templating
+- Deposit via S3 source pointer
+

ext/markdown

@@ -0,0 +1 @@
+Subproject commit 8c09109924b218aaecbfd4d4b1de538269c4d765

pocket_archive-scm-1.rockspec

@@ -35,6 +35,9 @@ dependencies = {
 build = {
     type = "builtin",
     modules = {
+        -- markdown is not set up for Lua 5.4 and cannot be installed via
+        -- Luarocks. Embedding the source here.
+        ["markdown"] = "ext/markdown/markdown.lua",
         ["pocket_archive"] = "src/core.lua",
         ["pocket_archive.model"] = "src/model.lua",
         ["pocket_archive.validator"] = "src/validator.lua",

src/core.lua

@@ -70,7 +70,7 @@ local M = {
   ext is optional, and is appended to the raw path.
 
   if no_create is not nil or false, the directory will not be checked for
-  existence or created.
+  existence or created. This is normally set for web paths.
 
   return: full file path, with the optional extension if provided.
 --]]

src/generator.lua

@@ -3,6 +3,7 @@ local datafile = require "datafile"
 local dir = require "pl.dir"
 local etlua = require "etlua"
 local json = require "cjson"
+local markdown = require "markdown"
 local path = require "pl.path"
 local pp = require "pl.pretty"
 
@@ -171,31 +172,54 @@ local function generate_coll(s, mconf)
     local child_ref, child_label, child_mconf
     while child_s do
         _, child_ref = next(repo.gr:attr(child_s, model.id_to_uri.ref))
+        -- Skip relationship with long description doc.
+        if repo.gr:contains(
+            triple.new(s, model.id_to_uri.long_description, child_ref)
+        ) then goto skip end
+        child_mconf = get_mconf(child_ref)
         --if not child_ref then child_ref = child_s end
         _, child_label = next(repo.gr:attr(child_s, model.id_to_uri.label))
         if not child_label then
-            _, child_label = next(repo.gr:attr(child_ref, model.id_to_uri.label))
+            if child_mconf.types.file then
+                _, child_label = next(repo.gr:attr(child_ref, model.id_to_uri.source_path))
+                child_label = path.basename(child_label.data)
+            else
+                _, child_label = next(repo.gr:attr(child_ref, model.id_to_uri.label))
+            end
         end
-        child_mconf = get_mconf(child_ref)
+        if child_label.data then child_label = child_label.data end
         table.insert(members, {
             icon = get_icon_url(child_mconf.lineage),
             tn = get_tn_url(child_s),
             href = pkar.gen_pairtree("/res", child_ref.data, ".html", true),
-            label = child_label.data,
+            label = child_label,
             ctype_label = child_mconf.label,
         })
+
+        ::skip::
         _, child_s = next(repo.gr:attr(child_s, model.id_to_uri.next))
     end
 
-    local title, description
+    local title, description, body_rel
     _, title = next(repo.gr:attr(s, model.id_to_uri.label))
     _, description = next(repo.gr:attr(s, model.id_to_uri.description))
+    _, body_rel = next(repo.gr:attr(s, model.id_to_uri.long_description))
+
+    local body
+    if body_rel then
+        local body_res_path
+        _, body_res_path = next(repo.gr:attr(body_rel, model.id_to_uri.archive_path))
+        local bfh = assert(io.open(body_res_path.data, "r"))
+        body = markdown(bfh:read("a"))
+        bfh:close()
+    end
 
     out_html = templates.coll.data({
         --webroot = M.webroot,
         site_title = pkar.config.site.title or pkar.default_title,
         title = title.data,
         description = description.data,
+        body = body,
         head_tpl = templates.head.data,
         header_tpl = templates.header.data,
         mconf = mconf,

src/util/watcher.lua

@@ -18,19 +18,19 @@ end)
 cli.locale "en_US"  -- TODO set with multilingual support.
 
 cli.program {
-    [[Run deposit watcher on a directory.
+    [[Run a laundry list watcher on a directory.
 
     Laundry list files dropped on the indicated directory will be processed
     for deposit.
 
-    NOTE: LL files MUST be dropped AFTER all files in the deposit have been
-    successfully uploaded.
+    NOTE: LL files MUST be dropped AFTER all files in the submission package
+    have been successfully uploaded.
     ]],
 
     cli.positional "path" {"Local path of folder to watch."},
 
     cli.flag "l,logfile" {
-        "Log file",
+        "Log file.",
         type = cli.string,
         default="/dev/stderr",
     },

templates/coll.html

@@ -22,8 +22,13 @@
                     <p><a href="#">Download Laundry List [TODO]</a></p>
                 </section>
             </div>
+            <% if body then %>
+            <section id="long_desc">
+                <%- body %>
+            </section>
+            <% end %>
             <% if #members > 0 then %>
-            <section id="res_mwmbers">
+            <section id="res_members">
                 <h2>Members</h2>
                 <ul class="slideshow">
                 <% for _, el in ipairs(members) do %>

templates/head_common.html

@@ -1,3 +1,4 @@
+<meta charset="utf-8"/>
 <!-- Milligram CSS -->
 <link rel="stylesheet" href="/css/normalize.min.css">
 <link rel="stylesheet" href="/css/milligram.min.css">

templates/ores.html

@@ -12,7 +12,7 @@
             </h1>
             <% if pres then %>
             <section id="res_pres">
-            <% if mconf.gen.presentation_type == "image" then %>
+                <% if (mconf.gen or {}).presentation_type == "image" then %>
                 <img src="<%= pres -%>" />
             <%else %>
                 <a href="<%= pres -%>" download="<%= fname -%>">

test/sample_submission/demo01/demo_collection/description.md

@@ -0,0 +1,26 @@
+## My Beautiful Pocket Archive Collection
+
+This collection is here to entertain the patient learners of Pocket Archive.
+
+It contains numerous wonderous delicierous featurous such as:
+
+- Bullet lists
+- Another bullet in the bullet list
+- And more free lines
+
+### Quote from an all-time favorite poem:
+
+> There I saw one I knew, and stopped him, crying: 'Stetson!  
+> ‘You who were with me in the ships at Mylae!  
+> ‘That corpse you planted last year in your garden,  
+> ‘Has it begun to sprout? Will it bloom this year?  
+> ‘Or has the sudden frost disturbed its bed?  
+> ‘Oh keep the Dog far hence, that’s friend to men,  
+> ‘Or with his nails he’ll dig it up again!  
+> ‘You! hypocrite lecteur!—mon semblable,—mon frère!”
+
+### More *Unicode* characters
+
+ستيفانو كوسو كان هنا
+
+

test/sample_submission/demo01/pkar_submission.csv

@@ -1,18 +1,19 @@
-"source_path","ext_id","content_type","label","alt_label","description","location","date","has_member","pref_rep"
-"demo_collection","coll0001","collection","My Demo Collection","My Beautiful  Collection","Some random stuff from my hard drive.",,2025-07-28,,"demo_collection/single_image/0685_04.jpg"
-,,,,"My Aunt's Beautiful Collection","Old B/W photos.",,,,
-,,,,,"More description to demonstrate how multi-valued fields are filled.",,,,
-,,,,,"""id"" fields have been left blank to let the system auto-generate them.",,,,
-"demo_collection/demo_postcard","art0001","postcard","Example Postcard","This is an alternative label","Note that recto and verso representations have been named front and back, to emphasize that the ordering is not alphabetical.",,2025-06-10,,
-,,,,,,,,,
-"demo_collection/demo_postcard/front",,"brick","Recto",,"A windy spring day in Capo Falcone, Sardinia","Capo Falcone (SS) Italy",2004-04-12,,
-,,,,,,"https://www.openstreetmap.org/#map=18/40.9696884/8.2020324",,,
-"demo_collection/demo_postcard/front/54321.jpg","file0001","still_image_file",,,,,,,
-"demo_collection/demo_postcard/back",,"brick","Verso",,"Wandering around somewhere in Tirana, 2006.","Tirana, Albania",2006-05-05,,
-,,,,,,"https://www.openstreetmap.org/relation/1250106",,,
-"demo_collection/demo_postcard/back/567890.jpg","file0002","still_image_file",,,,,,,
-"demo_collection/single_image","art0002","still_image","Preparing kebab at Aqil's during curfew",,"Nothing much to do under curfew but cooking, eating, singing, dancing, playing cards, smoking water pipe, and occasionally playing soccer in the street when the Merkava didn't get in the way.","Nablus, Palestine",2002-08-16,,
-,,,,,,"https://www.openstreetmap.org/#map=19/32.221597/35.260929",,,
-"demo_collection/single_image/0685_04.jpg","file0003","still_image_file",,,,,,,
-"coll2","coll0002","collection","Second collection",,"Collection made by reusing files from another collection.",,,"demo_collection/single_image/0685_04.jpg","demo_collection/single_image/0685_04.jpg"
-,,,,,,,,"demo_collection/demo_postcard/front/54321.jpg",
+"source_path","ext_id","content_type","label","alt_label","description","location","date","has_member","pref_rep","long_description"
+"demo_collection","coll0001","collection","My Demo Collection","My Beautiful  Collection","Some random stuff from my hard drive.",,2025-07-28,,"demo_collection/single_image/0685_04.jpg","demo_collection/description.md"
+,,,,"My Aunt's Beautiful Collection","Old B/W photos.",,,,,
+,,,,,"More description to demonstrate how multi-valued fields are filled.",,,,,
+,,,,,"""id"" fields have been left blank to let the system auto-generate them.",,,,,
+"demo_collection/demo_postcard","art0001","postcard","Example Postcard","This is an alternative label","Note that recto and verso representations have been named front and back, to emphasize that the ordering is not alphabetical.",,2025-06-10,,,
+,,,,,,,,,,
+"demo_collection/demo_postcard/front",,"brick","Recto",,"A windy spring day in Capo Falcone, Sardinia","Capo Falcone (SS) Italy",2004-04-12,,,
+,,,,,,"https://www.openstreetmap.org/#map=18/40.9696884/8.2020324",,,,
+"demo_collection/demo_postcard/front/54321.jpg","file0001","still_image_file",,,,,,,,
+"demo_collection/demo_postcard/back",,"brick","Verso",,"Wandering around somewhere in Tirana, 2006.","Tirana, Albania",2006-05-05,,,
+,,,,,,"https://www.openstreetmap.org/relation/1250106",,,,
+"demo_collection/demo_postcard/back/567890.jpg","file0002","still_image_file",,,,,,,,
+"demo_collection/single_image","art0002","still_image","Preparing kebab at Aqil's during curfew",,"Nothing much to do under curfew but cooking, eating, singing, dancing, playing cards, smoking water pipe, and occasionally playing soccer in the street when the Merkava didn't get in the way.","Nablus, Palestine",2002-08-16,,,
+,,,,,,"https://www.openstreetmap.org/#map=19/32.221597/35.260929",,,,
+"demo_collection/single_image/0685_04.jpg","file0003","still_image_file",,,,,,,,
+"coll2","coll0002","collection","Second collection",,"Collection made by reusing files from another collection.",,,"demo_collection/single_image/0685_04.jpg","demo_collection/single_image/0685_04.jpg",
+,,,,,,,,"demo_collection/demo_postcard/front/54321.jpg",,
+"demo_collection/description.md",,"file","Collection description",,,,,,,