Export not working properly

🧩 Low-Code Development
#1

We’ve recently upgraded to 2024.9.9 and it seems the export functionality is no longer working properly for all modules. It seems to work for some modules, but not others, and I can’t discern the difference. I’ll attach quite a few screenshots here in the hopes that helps.

Typically what happens is it’ll say there’s X number of records to export (which is correct). Then it will export and it will only include some smaller portion of records before it fails. Where it fails seems to be random which leads me to believe it’s not just some odd value in a record that’s causing it to break. The fact that it’s happening in different modules also leads me to think it’s some deeper issue.

image (2)
image (2)1147×937 98.8 KB

Here’s it showing only 4093 of those records exported. It ends with a unexpected operator ‘=’ at last node message.

image (3)
image (3)1062×161 16.5 KB

Also - often times the export just fails all together and we see this message.

image (4)
image (4)1062×743 40.8 KB

I’ve tried both csv and json export and getting the same result. We rely pretty heavily on the export feature so hoping to get some feedback soon! Thanks all!

#2

@emmanuel would you be able to provide with server logs when the export fails?

#3

I wasn’t able to reproduce this behavior I managed to export 34479 rows of 20 columns without issues with ID enabled or not.

I did run into an issue when exporting values that weren’t serializable by corteza (after importing badly formatted values) but that usually prevented the export altogether when testing 2024.9.9.

You could try exporting only the last record that corteza tried to export (your case named C2515) to see if it also fail.

#4

@emmanuel & @tjerman - We just updated to 9.9 and are experiencing the same thing. Did you guys identify a fix for this? TY!

#5

@kenderson Is the same happening on 2024.9.10?
As it is a hard-to-reproduce issue on our part, server logs would help debug this.

#6

@jfortun Yes, it still happens on 2024.9.10. We found the cause with the help of Claude. I work with @kenderson

The download request returns HTTP 500 with the body malformed expression, unexpected operator '=' at last node. The server log records nothing about it at LOG_LEVEL=warn, which is probably why logs haven’t helped.

Cause: in server/compose/envoy/record_datasource.go, resolveReferences runs for every Record-type field in the module, not just the exported columns. For each value it builds fmt.Sprintf("recordID=%s", v). If a Record field holds an empty string, the query becomes (recordID=) and the QL parser rejects it. The export streams records in batches of 100 with the response already started, which explains the partial files @emmanuel saw: it dies at the first batch that contains a blank reference.

To reproduce: in any module with a Record field, set that field’s stored value to "" on one record (an API update with {"name": "<field>", "value": null} does this), then export the module with any column selection.

Workaround: remove the empty-string values from Record fields. Here’s how to count them in a DAL-backed module:

SELECT COUNT(*) FROM cr__ WHERE JSON_SEARCH(values, ‘one’, ‘’, NULL, ‘$.’) IS NOT NULL;

Suggested fix: skip empty values in resolveReferences (and in the level-2 lookup) before building the recordID= terms.