feat: import sections, subsections, and units from TOML into Learning Core by dwong2708 · Pull Request #392 · openedx/openedx-core

dwong2708 · 2025-09-24T18:07:51Z

Resolves: #381

Description

This PR enables importing sections, subsections, and units into Learning Core from the contents of a TOML file.
This ensures that library structures can be consistently restored from backup definitions.

Changes

Implement restoration of sections, subsections, and units along with their child entities.
Improve data-saving logic by using bulk draft transactions.
Update publishing_api.create_next_container_version to handle cases where container.versioning.latest is None.
Add a new table/tag to identify the container type. See example below.

[entity]
uuid = "0b52444b-0604-4510-be71-960251e7343d"
can_stand_alone = true
key = "subsection1-48afa3"
created = 2025-09-04T22:51:52.824941Z

[entity.draft]
version_num = 2

[entity.published]
# unpublished: no published_version_num

[entity.container.subsection]

# ### Versions

[[version]]
title = "Subsection1"
uuid = "46642bce-0861-4e22-8b37-3cd897c99239"
version_num = 2

[version.container]
children = ["unit1-b7eafb"]

Input

test2.zip

Django Admin Output

Publishable entities

Draft Change Logs

Containers

Entity Lists

Entity List Detail for Unit

Entity List Detail for Subsection

Entity List Detail for Section

openedx-webhooks · 2025-09-24T18:07:55Z

Thanks for the pull request, @dwong2708!

This repository is currently maintained by @axim-engineering.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

ormsbee · 2025-09-25T19:10:32Z

+        try:
+            container = attrs["container"]
+            if "children" not in container:
+                raise ValueError("Missing 'children' in container")


These should raise serializers.ValidationError.

Fixed. Thank yout.

ormsbee · 2025-09-25T19:11:14Z

+            if not isinstance(children, list):
+                raise ValueError("'children' must be a list")


Generally, single-field validation is done with validate_<field_name> methods.

Changes applied. Thank you.

ormsbee · 2025-09-25T19:13:28Z

+            attrs["children"] = children
+            attrs.pop("container")  # Remove the container field after processing
+        except ValueError as exc:
+            raise serializers.ValidationError({"key": str(exc)})


Oh, I see, we're aggregating errors and re-raising. The nice thing about using separate validate_<field> methods is that it can surface multiple errors at the same time. This pattern you have here will short-circuit on the first error and hide the other ones.

Yeah, good catch. I've added those errors in an array. Thank you.

ormsbee · 2025-09-25T19:15:10Z

+        container_map = {
+            "section": "section",  # name of the container class : name of the toml table
+            "subsection": "subsection",
+            "unit": "unit",


I'm not clear what the purpose of this map is, if all the values map to themselves?

I thought the class name could be different from the TOML table value, but I changed it since it is the same value for now.

ormsbee · 2025-09-25T19:22:15Z

+        if self.errors:
+            raise ValueError(
+                "Errors encountered during restore: "
+                + "; ".join([f"{err['file']}: {err['errors']}" for err in self.errors])
+            )


Please definitely put this in a log file instead of raising a big ValueError..

Done. Thanks

ormsbee

Wrote some inline comments earlier, wanted to make sure you saw this.

ormsbee

Minor request in terms of testing the publishing api change.

Are you planning to make more tests for zip uploads, e.g. writing out subdirectories that represent test use cases and then making an in-memory ZipFile of them in order to test that it uploads the way you expect?

ormsbee · 2025-09-26T19:05:21Z

                entity.learning_package_id,
                last_version,
-                entity_rows,
+                entity_rows if entity_rows is not None else [],


Whenever you make an addition to another app's api.py file, please add a corresponding test for it. So in this case, please add a test case method that covers this alongside the other publishing api tests.

Understood. I added the test case for it. Thank you

dwong2708 · 2025-09-27T01:37:43Z

Minor request in terms of testing the publishing api change.

Are you planning to make more tests for zip uploads, e.g. writing out subdirectories that represent test use cases and then making an in-memory ZipFile of them in order to test that it uploads the way you expect?

Hi Dave, yes. I plan to include those tests in my final PR

ormsbee · 2025-09-29T00:41:47Z

Hi Dave, yes. I plan to include those tests in my final PR

Sorry, to clarify: Do you mean that you'll include those tests in the final version of this PR, or that you intend to add them in a follow-on PR?

dwong2708 · 2025-09-29T16:17:48Z

Hi Dave, yes. I plan to include those tests in my final PR

Sorry, to clarify: Do you mean that you'll include those tests in the final version of this PR, or that you intend to add them in a follow-on PR?

Oh, sorry for my previous message. I meant that I plan to add them in a follow-on PR

feat: include container type table in entity TOML output

fd12fe6

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Sep 24, 2025

openedx-webhooks added this to Contributions Sep 24, 2025

github-project-automation Bot moved this to Needs Triage in Contributions Sep 24, 2025

feat: implement container restoration and improve unzipper structure

1dcfad6

dwong2708 changed the title ~~feat: include container type table in entity TOML output~~ feat: import sections, subsections, and units from TOML into Learning Core Sep 25, 2025

dwong2708 requested a review from ormsbee September 25, 2025 17:12

dwong2708 marked this pull request as ready for review September 25, 2025 17:12

mphilbrick211 added the mao-onboarding Reviewing this will help onboard devs from an Axim mission-aligned organization (MAO). label Sep 25, 2025

mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Sep 25, 2025

ormsbee reviewed Sep 25, 2025

View reviewed changes

feat: add log file for errors and improve serializer validations

2a6ba98

ormsbee requested changes Sep 25, 2025

View reviewed changes

dwong2708 requested a review from ormsbee September 25, 2025 22:45

ormsbee requested changes Sep 26, 2025

View reviewed changes

test: add test for create_next_container_version API

d95fa7c

dwong2708 requested a review from ormsbee September 27, 2025 01:37

ormsbee approved these changes Sep 29, 2025

View reviewed changes

ormsbee merged commit ce359a2 into openedx:main Sep 29, 2025
11 checks passed

github-project-automation Bot moved this from Ready for Review to Done in Contributions Sep 29, 2025

ormsbee mentioned this pull request Dec 30, 2025

Unify smaller apps into one openedx_content app #454

Merged

		if not isinstance(children, list):
		raise ValueError("'children' must be a list")

Conversation

dwong2708 commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Input

Django Admin Output

Uh oh!

openedx-webhooks commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ormsbee left a comment

Choose a reason for hiding this comment

Uh oh!

ormsbee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dwong2708 commented Sep 27, 2025

Uh oh!

ormsbee commented Sep 29, 2025

Uh oh!

dwong2708 commented Sep 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dwong2708 commented Sep 24, 2025 •

edited

Loading

openedx-webhooks commented Sep 24, 2025 •

edited

Loading