I’ve been poking around the CSV export function trying to figure out all the data types and so on (to build a parser), but the changes I’ve made on my profile aren’t showing up, they only changed the first time I did it today. Are the later emails perhaps sending me a cached version of the .csv? When will the cache entry be invalidated?
Better yet, is there a spec somewhere for all the various fields’ data types and encodings?
It probably is a cloudfront cache thing. I’ll probably have to version the files to bust the cache.
As for the spec, we don’t have anything written out. I don’t know if I’d remember to keep it up to date to be honest.
The structured values are all json. The rest are pretty obvious. I have a Power BI Desktop file that parses content from the csv if you want to play around with some charts and tables. Power BI Desktop is a free download from Microsoft (I work on the team that builds it).
Ok that’s fine, I think I got all the models and encodings covered, will post the code this week, I’ve finished the parsing and am on to doing the output I need.
Whenever there’s a json model in quotes I replace the escaped double double quotes with singular double quotes, get rid of the surrounding double quotes before throwing it into a Json parser which seems to handle escaped unicode characters etc., let me know if you know any steps I’m missing or doing wrong.
Yes they’re Json, but to read the Json I need to know the field names and how the models are set up etc. They’re also escaping at least double quotes in a way I haven’t seen before, and I don’t know if I’m missing some other characters I need to handle.
Power Bi looks interesting though, I’ll have to look into it, I was just going to output a CSV and read it in Excel.
Yeah the escaping is funky. Make sure you’re using UTF-8 encoding to open the file.