Errors processing CP2K DFT data while using dpdata: Length of atom_numbs is 1, but expected 1 #354

lmqnuaa · 2022-09-02T08:15:01Z

lmqnuaa
Sep 2, 2022

Hello, I met an error while using dpdata to process MD data from CP2K(7.0 Development version):
`>>> import dpdata

dpdata.LabeledSystem('../graphene_DFT',fmt='cp2k/aimd_output').to('deepmd/npy','data1',set_size=5)`

Traceback (most recent call last):
File "", line 1, in
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 226, in init
self.from_fmt(file_name, fmt, type_map=type_map, begin= begin, step=step, **kwargs)
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 252, in from_fmt
return self.from_fmt_obj(load_format(fmt), file_name, **kwargs)
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 1016, in from_fmt_obj
self.append(LabeledSystem(data=dd))
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 222, in init
self.check_data()
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 242, in check_data
dd.check(self)
File "/home/lmq/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 125, in check
raise DataError("Length of %s is %d, but expected %d" % (self.name,
dpdata.system.DataError: Length of atom_numbs is 1, but expected 1

I put the .log file and pos.xyz file in the same folder named "graphene_DFT". The files are attached below.

graphene_DFT.zip

Answered by robinzyb

Sep 28, 2023

@SchrodingersCattt @PolyuWeldingSpock Update for parsing cells from npt ensemble, please check the devel branch of cp2kdata

import dpdata
cp2kmd_dir = "./npt/"
cp2kmd_output_name = "cp2k.log"
dp = dpdata.LabeledSystem(cp2kmd_dir, cp2k_output_name=cp2kmd_output_name, fmt="cp2kdata/md")
print(dp)

--- You are parsing data using package Cp2kData ---
Parsing Energies From ./npt/cp2k-1.ener
Parsing Structures From ./npt/cp2k-pos-1.xyz
Parsing Forces from the CP2K output/log file: ./npt/cp2k.log
Parsing Cells Information from ./npt/cp2k.log
Atom names are fake chemical symbols as you set in cp2k input.
--- You are parsing data using package Cp2kData ---
Data Summary
Labeled System
--------------…

View full answer

HuangJiameng · 2022-09-23T02:02:49Z

HuangJiameng
Sep 23, 2022
Collaborator

#351 Maybe this can help you?

0 replies

HuangJiameng · 2022-09-23T02:12:21Z

HuangJiameng
Sep 23, 2022
Collaborator

You can check if the log is in line with the template in https://github.com/deepmodeling/dpdata/blob/master/dpdata/cp2k/output.py#L101

0 replies

HuangJiameng · 2022-09-23T03:58:05Z

HuangJiameng
Sep 23, 2022
Collaborator

When extracting atom_numb, dpdata finds the keyword "ATOMIC FORCES in" to read force lines and get information about atoms. But there are no "ATOMIC FORCES in" in your log. Please check it.

0 replies

SchrodingersCattt · 2023-09-11T05:15:15Z

SchrodingersCattt
Sep 11, 2023

May I ask whether your problem was solved then and how? I encountered the same error today, and I have checked my log file to contain ATOMIC FORCES information.

7 replies

SchrodingersCattt Sep 11, 2023

Hello, thank you for your patient reply. You might have checked the log file from lmqnuaa's posted log file above.

My inp file and log file are attached and I did used the MEDIUM level for parsing. Actually I conveyed a series of CP2K simulations with changing the coordinates only. Other results were successfully converted to the format for DeepMD, except this. I would appreciate it if you could have a check. Thanks for your time!
aimd.tar.xz.zip

robinzyb Sep 11, 2023
Maintainer

Hello, do you have -frc-, -pos-,.ener files? fmt=cp2k/aimd_output does not support version like 2023.1. It only supports for cp2k/output. In addition, using fmt=cp2k/aimd_output requires the -pos- file which I didn't find.
For me, I recommend cp2kdata since it can parse all of these, both MD and single point energy calculation.

SchrodingersCattt Sep 11, 2023

I have the .ener and -pos- files and do not have -frc-, which the xyz file was a bit large to upload together just now. Now the two files are attached. Kindly have a check.

For the format issue, I utilized the output.py modified by #498, which fits version 2023.1 well.

I will try cp2kdata, thank you!

aimd-xyz_ener.zip

robinzyb Sep 11, 2023
Maintainer

For the format issue, I utilized the output.py modified by #498, which fits version 2023.1 well.

does it fit the problem of current 2023.1 version as you provided? If so, I will not check them again.

For MD output of -frc- files, please write these in the MOTION section

&MOTION
  &PRINT
   &TRAJECTORY
     &EACH
       MD 1
     &END EACH
   &END TRAJECTORY
   &FORCES 
     &EACH
       MD 1
     &END EACH
   &END FORCES
  &END PRINT
&END MOTION

These are common practice of using cp2k AIMD. For single point energy calculation, both cp2k/output and cp2kdata/e_f work well. Just be careful for the MD output, in which you have flexibility to output variable format. You can have a test for 2 or 3 steps AIMD and use cp2kdata to parse it.

SchrodingersCattt Sep 11, 2023

does it fit the problem of current 2023.1 version as you provided?

No, I can parse other outputs except this and that's why I feel weird. I do not think this error is caused by version issue.

robinzyb · 2023-09-11T20:51:41Z

robinzyb
Sep 11, 2023
Maintainer

Hi @SchrodingersCattt , I update cp2kdata (robinzyb/cp2kdata@163b5d0). Please switch to the devel branch(https://github.com/robinzyb/cp2kdata/tree/devel) of cp2kdata repository and pip install it.

However, it may be unstable because I found that your log file terminated without a normal ending. Please set WALLTIME under the GLOBAL section to a time slightly shorter than the running time of the job scheduler so that CP2K can write the log and other files completely.

In addition, it would be better if you can rerun 2 steps of current md and return me the output files as my testing files of cp2kdata. Thank you.

Test of your `cp2k.log`

import dpdata
cp2kmd_dir = "./data/"
cp2kmd_output_name = "cp2k.log"
dp = dpdata.LabeledSystem(cp2kmd_dir, cp2k_output_name=cp2kmd_output_name, fmt="cp2kdata/md")
print(dp)

--- You are parsing data using package Cp2kData ---
You are reading cell information from ./data/cp2k.log
Parsing Energies From ./data/cp2k-1.ener
Parsing Structures From ./data/cp2k-pos-1.xyz
Parsing Forces from the CP2K output/log file: ./data/cp2k.log
Atom names are fake chemical symbols as you set in cp2k input.
--- You are parsing data using package Cp2kData ---
Data Summary
Labeled System
-------------------
Frame Numbers      : 4986
Atom Numbers       : 174
Including Virials  : No
Element List       :
-------------------
I  Rb  N  H  C
18  6  12  96  42

7 replies

SchrodingersCattt Sep 25, 2023

No, it doesn't.

For one thing, I tried to parse the AIMD with NpT ensemble, but the content of box.npy gives the same values. Actually, the very same problem occurred when I used only dpdata for parsing.
(The files are a bit large so I can't upload as an attachment, would you please leave an email? or I will try a simpler NPT AIMD tomorrow.) The output.py was switched to the original one of dpdata other than the modified one I mentioned above. (both files give the same box.npy result)

For another, I just tried another totally different system containing different elements, and it stuck at the step of parsing .xyz file. (the .xyz file is a bit large so let me wait overnight and see if it can smoothly work.)

Thanks for your time!

robinzyb Sep 25, 2023
Maintainer

There is no support for parsing npt trajectory for now. I haven't run it. What is the best practice for outputing cell information of cp2k with npt trajectory?

SchrodingersCattt Sep 26, 2023

I found the Cell information in the .log files of every frame for NpT AIMD. I think it is technically feasible to parse that. Would you please consider add this NpT-parsing function in the future updates? Or maybe do you know any other cp2k-dp format-converting scripts?

robinzyb Sep 26, 2023
Maintainer

I would like to implement it npt. Could you provide a very short trajectory, 3 steps, for me to enhance cp2kdata? The short trajectory helps me check the bug when developing the code.

SchrodingersCattt Sep 26, 2023

NpT_and_NVT.zip
Hi, I have re-run two 3-step AIMDs with CP2K 2023.1 in both NpT and NVT. Kindly have a check. Thanks!

robinzyb · 2023-09-26T08:44:34Z

robinzyb
Sep 26, 2023
Maintainer

Sorry for the late reply. I tested on another data set and succeeded. Here are the re-run results. Kindly have a check. Thank you very much for your patient help! cp2k-pos-1.zip

I downloaded it, it seems contain 1550 steps. it too large to put in cp2kdata codes. Could you only provide the trajectory with 2-3 steps and with a normal ending of a cp2k log file

2 replies

robinzyb Sep 28, 2023
Maintainer

@SchrodingersCattt @PolyuWeldingSpock Update for parsing cells from npt ensemble, please check the devel branch of cp2kdata

import dpdata
cp2kmd_dir = "./npt/"
cp2kmd_output_name = "cp2k.log"
dp = dpdata.LabeledSystem(cp2kmd_dir, cp2k_output_name=cp2kmd_output_name, fmt="cp2kdata/md")
print(dp)

--- You are parsing data using package Cp2kData ---
Parsing Energies From ./npt/cp2k-1.ener
Parsing Structures From ./npt/cp2k-pos-1.xyz
Parsing Forces from the CP2K output/log file: ./npt/cp2k.log
Parsing Cells Information from ./npt/cp2k.log
Atom names are fake chemical symbols as you set in cp2k input.
--- You are parsing data using package Cp2kData ---
Data Summary
Labeled System
-------------------
Frame Numbers      : 4
Atom Numbers       : 116
Including Virials  : No
Element List       :
-------------------
I  Rb  N  H  C
12  4  8  64  28

dp['cells']

array([[[14.755     ,  0.        ,  0.        ],
        [ 0.725     ,  8.168     ,  0.        ],
        [ 1.149     ,  1.049     , 12.907     ]],

       [[14.7585631 ,  0.        ,  0.        ],
        [ 0.72251471,  8.16856195,  0.        ],
        [ 1.14795782,  1.04800923, 12.90595636]],

       [[14.7622873 ,  0.        ,  0.        ],
        [ 0.72040634,  8.16905262,  0.        ],
        [ 1.14712155,  1.04670322, 12.90584462]],

       [[14.76615   ,  0.        ,  0.        ],
        [ 0.7182691 ,  8.16942772,  0.        ],
        [ 1.1463185 ,  1.04542975, 12.90611961]]])

Answer selected by robinzyb

SchrodingersCattt Sep 29, 2023

Thank you very much. The update helps a lot.
Wish you happy Mid-Autumn and National Day!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Errors processing CP2K DFT data while using dpdata: Length of atom_numbs is 1, but expected 1 #354

{{title}}

Replies: 6 comments 16 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Errors processing CP2K DFT data while using dpdata: Length of atom_numbs is 1, but expected 1 #354

Replies: 6 comments · 16 replies

HuangJiameng Sep 23, 2022 Collaborator

HuangJiameng Sep 23, 2022 Collaborator

HuangJiameng Sep 23, 2022 Collaborator

robinzyb Sep 11, 2023 Maintainer

robinzyb Sep 11, 2023 Maintainer

robinzyb Sep 11, 2023 Maintainer

Test of your cp2k.log

robinzyb Sep 25, 2023 Maintainer

robinzyb Sep 26, 2023 Maintainer

robinzyb Sep 26, 2023 Maintainer

robinzyb Sep 28, 2023 Maintainer

Replies: 6 comments 16 replies

HuangJiameng
Sep 23, 2022
Collaborator

HuangJiameng
Sep 23, 2022
Collaborator

HuangJiameng
Sep 23, 2022
Collaborator

robinzyb Sep 11, 2023
Maintainer

robinzyb Sep 11, 2023
Maintainer

robinzyb
Sep 11, 2023
Maintainer

Test of your `cp2k.log`

robinzyb Sep 25, 2023
Maintainer

robinzyb Sep 26, 2023
Maintainer

robinzyb
Sep 26, 2023
Maintainer

robinzyb Sep 28, 2023
Maintainer