The results do not agree with those in the paper

[yxt7979]

Hi, thank you for your excellent work!
When I try to reproduce the experiments in the paper, I can't get the same score as in the paper for 8 scenes.
Like scene0553, the rgb score is :

While I can only get the score below after 200001 iterations :

In my experiment, the hyperparameter Setting is the same as the code in the project:

May I ask if you can inform me the Hyperparameter Setting of the experiment with the score in the paper, as we are interested in reproducing the excellent experimental results! Thank you very much!

[yxt7979]

Hi! We use the same code of https://github.com/weiyithu/NerfingMVS/blob/main/src/evaluation.py , all the code is as the same as the project on the github.

…

------------------ 原始邮件 ------------------ 发件人: "weiyithu/NerfingMVS" ***@***.***>; 发送时间: 2022年8月24日(星期三) 中午12:57 ***@***.***>; ***@***.******@***.***>; 主题: Re: [weiyithu/NerfingMVS] The results do not agree with those in the paper (Issue #36) Hi, may I ask which evlauation code do you use to print the depth metrics? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[Beniko95J]

Hi, thank you for telling me about this. I just found that there was a file named 'evaluation.py' in the project so I just deleted my question. Never mind!

[weiyithu]

hi, thanks for your attention. Do you use this config file https://github.com/weiyithu/NerfingMVS/blob/main/configs/scene0553_00.txt?

[yxt7979]

yes！in the command line

…

---Original--- From: "Yi ***@***.***> Date: Wed, Aug 24, 2022 17:18 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [weiyithu/NerfingMVS] The results do not agree with those in thepaper (Issue #36)

[weiyithu]

It is a little bit strange. Have you tried our pretrained model or training from scratch on other scenes?

[yxt7979]

Hi! We trained for all the eight scenes, while the results in each scene are less than that of the paper, the results are shown as below:

We train the model from the beggining ( colmap -> mono depth network -> nerf ) using 'sh run.sh' , and we found the checkpoints are the final checkpoints for 8 scenes.

We didn't load any pretrained model in our experience, could you please tell me are there any pretrained models we need to load while training? Or maybe there's something wrong with our hyperparameter settings according to the options.py?

And another question! Will I get different test results every time I train for the same scene?

Thank you for your response! We meant no harm but only want to get a good rendering! Thank you again and looking forward to your reply!

[weiyithu]

Hi, I have reproduced scene0553 and get right results:

Some things you should notice:

1. Did you use our COLMAP version? Such as struct .error: unpack requires a buffer of 12 bytes #2
2. Have you checked your python evironment? You can use our uploaded checkpoint to check this. If you still cannot reproduce results with our uploaded checkpoint, your python evironment may have problem.