Could you release the HLE benchmark evaluation code?

[xiaolin-cs]

Thank you for your great work on InternAgent. The project demonstrates strong performance on reasoning benchmarks such as HLE, which is very impressive.

However, I was not able to find the evaluation code or scripts for running InternAgent on HLE (or similar reasoning benchmarks) in the repository. It would be extremely helpful if the team could share the corresponding evaluation pipeline, including data preprocessing, prompt templates, and inference settings, to facilitate reproducibility and further research.

[Shiyang980713]

Thank you very much for your interest in InternAgent and for the thoughtful question! We truly appreciate your attention to our work and your interest in the evaluation details. 🙏

Regarding access and evaluation:

• We will soon open API access so that everyone can directly experience and experiment with our leading deep research capabilities.
• For HLE and other related benchmarks, we follow the official evaluation pipeline provided by the benchmark authors, which is also a common practice in the community to ensure fairness and comparability. In particular, for HLE we use the official repository at centerforaisafety/hle.

You are very welcome to try our API once it becomes available, and we hope it will help facilitate further research and reproducibility efforts. We would also like to note that running evaluations on these datasets can be quite costly.

Thanks again for your comments, and happy Chinese New Year! 🧧

Best,
InternAgent Team

[Shiyang980713]

Hi, @xiaolin-cs

Thank you again for your interest in our work! We noticed that you have open-sourced several excellent AI algorithm projects. If you’re interested, we would warmly welcome you to submit your task to InternAgent by opening an issue.

We will further optimize based on the baseline you provide and publicly share the corresponding results.

Looking forward to your reply!

Best,
InternAgent Team

[xiaolin-cs]

@Shiyang980713 Hi, it’s exciting to see the rapid progress of the InternAgent project — congratulations to the team on the recent developments!

I am a CS PhD student currently working on deep research and agentic systems, and I’m very interested in understanding the implementation details behind InternAgent 1.5. I was wondering whether there are any plans to release the source code for InternAgent 1.5, rather than providing access solely through the API.

[VigneshHexo]

@Shiyang980713, could you confirm if there are any plans to release the source code for InternAgent 1.5, rather than providing access solely through the API. Thank you very much

[Shiyang980713]

Hi all,

Thank you again for your continued interest in InternAgent and for the thoughtful follow-ups!

We’re happy to share that we have now opened API access to our Deep Research capability. It can be accessed via the SCP protocol, which is compatible with MCP. You can find detailed instructions and usage guidance here:

👉 https://scphub.intern-ai.org.cn/detail/28

We hope this will make it easier for researchers to experiment directly within their own workflows, as well as reproduce our reported performance.

Regarding open-sourcing InternAgent 1.5, we are currently undergoing internal review. We will share updates once there is progress. Thank you for your interest and patience.

Best,
InternAgent Team