Which data files are we not using yet? #348
Comments
lynxlynxlynx
added
feature
|
Did the job for the 2DA files without checking NI again. Steps to repeat: # Going through code by some patterns
$ ag --nocolor --nofilename --noheading --nonumbers -i "\sAutoTable" | awk -F ':' '{ print $2 }' | grep -oP "\"[[:alnum:]_]+\"" | sed 's/"//g' | sort | uniq >> findings.csv
$ ag --nocolor --nofilename --noheading --nonumbers -i "AutoTable\(" | awk -F ':' '{ print $2 }' | grep -oP "\"[[:alnum:]_]+\"" | sed 's/"//g' | sort | uniq >> findings.csv
$ ag --nocolor --nofilename --noheading --nonumbers -i "tm.load" | awk -F ':' '{ print $2 }' | grep -oP "\"[[:alnum:]_]+\"" | sed 's/"//g' | sort | uniq >> findings.csv
$ ag --nocolor --nofilename --noheading --nonumbers -i "LoadTable\s?\(" | awk -F ':' '{ print $2 }' | grep -oP "\"[[:alnum:]_]+\"" | sed 's/"//g' | sort | uniq >> findings.csv
# Should have done as well:
$ ag --nocolor --nofilename --noheading --nonumbers -i "ReadAbilityTable\s?\(" | awk -F ':' '{ print $2 }' | grep -oP "\"[[:alnum:]_]+\"" | sed 's/"//g' | sort | uniq >> findings.csv
# Normalization
$ cat findings.csv | awk '{ print tolower($0) }' | sort | uniq > findings_sorted.csv
$ cat lists.csv | awk '{ print tolower($0) }' | sed 's/^\s//' | sort | uniq > lists_sorted.csv
# Outer join
$ join -t, -a 1 -a 2 -j 1 -e NULL -o 1.1,2.1 lists_sorted.csv findings_sorted.csv > joined.csv
# Manual work
# - mark 'x' what I could identify manually
# - mark '?' what is indirectly referenced somewhere
# Filter out what remains NULL without manual resolution
$ awk -F ',' '$2 == "NULL" && $3 != "x" && $3 != "?" { print $1 }' joined.csvEt voilà: Click to expand |
|
Cool, good start. There's still plenty of false positives there though, like animations, xl3000, the npc epilogue files, enumerated tables like chapter text and some of the xxx brevities. reptext was a bug in iesdp, which I've now fixed. |
|
Maybe you can edit the comment to strike through the known false positives? e.g. no idea where you got xl3000 from but if you do. |
|
I'd make a new one, so the script output is preserved. But I opened this, so someone else could do it instead. ;) xl3000 is referenced by xnewarea.2da, the epilogues are referenced by TextScreen actions and things with numbers or underscores are often generated. |
|
With removed known users or where there are already reports:
|
|
IDS are different, since they're directly used in scripts and dialog as named constants. Still, to be complete, better check and perhaps something can be learned. Checking IESDP ( Clown color zoo (we load "clowncol"):
Misc todo:
Obviated:
Probably useless:
For many of them IESDP can be checked (actions/triggers), since they're linked in their param list. |
lynxlynxlynx
removed
the
good first issue
The original games came with a bunch of 2da tables and similar ids files. I wonder which we don't use yet. In the past it helped resolving what was eventually #89 for example.
This task is about searching our codebase and comparing — which files are we not referencing yet? Many 2das are for animations (4-letter, starting with a or m, like acat) and some others may also not show up, but I'd like to see a list of the outliers. This will eventually also help with #164.
2da:
https://gibberlings3.github.io/iesdp/files/2da/index.htm
(at minimum for pst and iwd2 it also makes sense to check with NI or DLTCEP, in case the listing is incomplete)
ids:
https://gibberlings3.github.io/iesdp/files/ids/index.htm
This will also help with Gibberlings3/iesdp#6.
Up to date lists:
The text was updated successfully, but these errors were encountered: