scossu/volksdata
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

Delete yields wrong results #5

New issue
Closed
opened 2026-01-17 06:24:26 +00:00 by scossu · 4 comments
scossu commented 2026-01-17 06:24:26 +00:00
Owner
Copy link

Test at https://git.knowledgetx.com/scossu/volksdata/src/branch/master/test/test_graph.c#L585 fails. This means the the VOLK_graph_delete function of he MDB store doesn't delete all the triples when passed zero bound terms.

The same issue seems to affect the VOLK_graph_lookup function as well.

The randomized triples reveal that multiple runs of the same tests fail by different counts.

This is likely the direct cause of scossu/volksdata_lua#3 .

Test at https://git.knowledgetx.com/scossu/volksdata/src/branch/master/test/test_graph.c#L585 fails. This means the the `VOLK_graph_delete` function of he MDB store doesn't delete all the triples when passed zero bound terms. The same issue seems to affect the `VOLK_graph_lookup` function as well. The randomized triples reveal that multiple runs of the same tests fail by different counts. This is likely the direct cause of https://git.knowledgetx.com/scossu/volksdata_lua/issues/3 .
scossu commented 2026-01-17 15:41:42 +00:00
Author
Owner
Copy link

Excerpt from volksdata_lua test log:

10:34:59 DEBUG src/graph.c:467: Removing triples by terms: <nil> <nil> <nil>
10:34:59 DEBUG src/store_mdb.c:241: BEGIN RW transaction 0x59f184367910 child of (nil) in env /tmp/volksdata_test_store.db
10:34:59 DEBUG src/store_mdb.c:678: Lookup context: c57b99fed7ebe69f
10:34:59 DEBUG src/store_mdb.c:1519: Looking up 0 bound terms.
10:34:59 DEBUG src/store_mdb.c:1543: Found 10 triples.
10:34:59 DEBUG src/store_mdb.c:1079: Found 10 triples to remove.
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 39e62ea889c84b04}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 39e62ea889c84b04}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 39e62ea889c84b04}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 236ebdcf525931b}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 236ebdcf525931b}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 236ebdcf525931b}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 1ed41faf78e42a3}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 1ed41faf78e42a3}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 1ed41faf78e42a3}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 759b99c08a88db42}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 759b99c08a88db42}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 759b99c08a88db42}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, ad8ebf3c7779f5d8}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, ad8ebf3c7779f5d8}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 ad8ebf3c7779f5d8}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, a4a5a55ff962de72}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, a4a5a55ff962de72}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 a4a5a55ff962de72}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 5793b26eecb163ce}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 5793b26eecb163ce}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 5793b26eecb163ce}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, dd191524c0063759}
10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f
10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, dd191524c0063759}
10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 dd191524c0063759}
10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE
10:34:59 DEBUG src/store_mdb.c:250: COMMIT transaction 0x59f184367910

10 triples are found in the summary, only 8 are handled in the iterator loop. It looks like it_next_0bound or something else is skipping, or returning too early, or yielding some invalid results that are silently failing.

Excerpt from volksdata_lua test log: ``` 10:34:59 DEBUG src/graph.c:467: Removing triples by terms: <nil> <nil> <nil> 10:34:59 DEBUG src/store_mdb.c:241: BEGIN RW transaction 0x59f184367910 child of (nil) in env /tmp/volksdata_test_store.db 10:34:59 DEBUG src/store_mdb.c:678: Lookup context: c57b99fed7ebe69f 10:34:59 DEBUG src/store_mdb.c:1519: Looking up 0 bound terms. 10:34:59 DEBUG src/store_mdb.c:1543: Found 10 triples. 10:34:59 DEBUG src/store_mdb.c:1079: Found 10 triples to remove. 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 39e62ea889c84b04} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 39e62ea889c84b04} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 39e62ea889c84b04} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 236ebdcf525931b} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 236ebdcf525931b} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 236ebdcf525931b} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 1ed41faf78e42a3} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 1ed41faf78e42a3} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 1ed41faf78e42a3} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 759b99c08a88db42} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 759b99c08a88db42} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 759b99c08a88db42} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, ad8ebf3c7779f5d8} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, ad8ebf3c7779f5d8} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 ad8ebf3c7779f5d8} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, a4a5a55ff962de72} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, a4a5a55ff962de72} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 a4a5a55ff962de72} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 5793b26eecb163ce} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, 5793b26eecb163ce} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 5793b26eecb163ce} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 TRACE src/store_mdb.c:797: Found spok: {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, dd191524c0063759} 10:34:59 TRACE src/store_mdb.c:807: Triple found in context: d7ebe69f 10:34:59 TRACE src/store_mdb.c:1088: Removing {6cf49ca23586c9a1, 12d1f0cb6b0e43e6, dd191524c0063759} 10:34:59 TRACE src/store_mdb.c:1316: Indexing triple: {6cf49ca23586c9a1 12d1f0cb6b0e43e6 dd191524c0063759} 10:34:59 TRACE src/store_mdb.c:1320: Indexing op: REMOVE 10:34:59 DEBUG src/store_mdb.c:250: COMMIT transaction 0x59f184367910 ``` 10 triples are found in the summary, only 8 are handled in the iterator loop. It looks like `it_next_0bound` or something else is skipping, or returning too early, or yielding some invalid results that are silently failing.
scossu commented 2026-01-17 18:01:41 +00:00
Author
Owner
Copy link

This cannot be reproduced for lookup, in any combination, with up to 10000 random triples.

On the other hand, it can be reproduced also by binding one term common to all triples, if the size of the graph is large enough (about 1 chance in 10-15 for 100 triples, almost always for 1000 triples).

It may be rather a problem with the position of the pointer after delete. There is a relocation function to reposition the pointer, but it may be flawed in a way that it only fails in these restricted cases.

This cannot be reproduced for lookup, in any combination, with up to 10000 random triples. On the other hand, it can be reproduced also by binding one term common to all triples, if the size of the graph is large enough (about 1 chance in 10-15 for 100 triples, almost always for 1000 triples). It may be rather a problem with the position of the pointer after delete. There is a relocation function to reposition the pointer, but it may be flawed in a way that it only fails in these restricted cases.
scossu changed title from Delete (and lookup?) yield wrong results to Delete yields wrong results 2026-01-18 18:11:22 +00:00
scossu commented 2026-01-18 18:12:11 +00:00
Author
Owner
Copy link

Not resolved. Parallel or separate transactions may be the way to go but I haven't found a way to make them work in the delete context yet.

Not resolved. Parallel or separate transactions may be the way to go but I haven't found a way to make them work in the delete context yet.
scossu reopened this issue 2026-01-18 18:12:11 +00:00
scossu commented 2026-01-22 22:47:17 +00:00
Author
Owner
Copy link

Some edge cases, caused by a bug in the 0-bound lookup, have been fixed too.

Some edge cases, caused by a bug in the 0-bound lookup, have been fixed too.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.0b7
v1.0b6
v1.0b5
v1.0b4
v1.0b3
v1.0b2
v1.0a4
v1.0a5
1.0a4
Labels
Clear labels   
t
bug

  
t
documentation

  
t
enhancement

  
t
feature

  
t
test

No labels
t
bug
t
documentation
t
enhancement
t
feature
t
test
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
scossu/volksdata#5
No description provided.