Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PTT version converter initial pass at ptt converter #600

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DSMishler
Copy link
Collaborator

Usage:

python ptt_converter.py 2t1 2023-05-01_xsdk_ts200_N4800_dtd_lfq_normal_tv2.prof.h5 test2v1.prof.h5

I ran into an issue with this a while ago but never made a PR to start the conversation. Currently the biggest issue is that converting backwards (from version 2 to version 1) seems to cause all the datatypes in all the columns to become strings. Now, that was something I had to do to get a workaround for an error if I didn't do it, but surely there must be a better way. For the time being, here is the tool for people to review and comment on.

@omor1 you're a python guru and the author of trace version 2 (which my trace tools have migrated to) and I'm sure you would have lots of helpful feedback. If you want to spare the time, please feel free to comment here or message me about it.

@DSMishler DSMishler requested a review from a team as a code owner November 20, 2023 19:07
@omor1
Copy link
Contributor

omor1 commented Nov 20, 2023

The problem is probably that the v1 format has a giant table with a bunch of empty entries. Since integer-typed columns don't have a way to represent "no value" (unlike float, which can use NaN), it does something silly and converts everything to object—or to string, I guess? Anyway, the point is that types were inherently a problem in the v1 format because of this, and is part of the reason why I created the v2 format.

@DSMishler
Copy link
Collaborator Author

Well, good to know. It seems like we were running into the exact same issue with converting to "object" or "string". This makes converting between the ptt versions a bit tenuous since realistically there doesn't seem to be a great way to convert between them, just a "good enough" way. The main goal is just to make it work with my visualization tool, and that at least seems doable.

@omor1
Copy link
Contributor

omor1 commented Nov 22, 2023

Look into pandas.DataFrame.infer_objects() to see if that can help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants