Feature request: Support for retaining image source documents

[whaynes]

Rob asked me to make an issue here requesting CLI support for retaining the drawing files from CAD, Inkscape, Illustrator, etc. somewhere in the source tree so that they can be accessed as needed to update and regenerate the assets made with them.

I keep all my original files alongside the generated assets in a 'resources' folder Before building the book, I run this script (thanks, Stephen) that recursively searches the folder for specific filetypes, and moves them to the right places. Presumably, the source folder, destination folder and filetypes would be specified in project.ptx.

https://github.com/dantheboatman/EngineeringStatics/blob/master/scripts/update_assets.py

I think this would be generally useful to many, and it would be nice to have as part of the build process because its easy to forget to run the script.

[StevenClontz]

It's up to Oscar, but unless core PreTeXt is responsible for generating the assets (e.g. a <latex-image/> or <sageplot/> or <prefigure/>), I think it's out of scope for the CLI to do anything here.

I would suggest instead that you write a custom Python build script that calls PreTeXt "CLI" as a Python library. Here's an example of what we do at TBIL.org to support our workflow that goes beyond the scope of PreTeXt itself: https://github.com/TeamBasedInquiryLearning/library/blob/StevenClontz/20250723/edition-prep/scripts/prepare_edition.py In particular, you shouldn't forget to run your script, because you should always be running your script rather than invoking the vanilla CLI directly if your project is beyond the scope of vanilla PreTeXt. My two cents.

[rbeezer]

Will said:

I run this script (thanks, Stephen) that recursively searches the folder for specific filetypes, and moves them to the right places.

Steven said:

unless core PreTeXt is responsible for generating the assets

I was thinking more of the former, and not the latter. It would be the author's job to run whatever tools they like (CAD software, Inkscape, etc) to create PNG, SVG, etc. Then the CLI would simply copy those to some isomorphic location within the external directory where PreTeXt would find them as usual.

The author would include in their project file some sort of specification like

<project-root>/cad/chapter-intro/*.png

and then thier PreTeXt source could have

<figure>
    <image source="cad/chapter-intro/ship.png"/>
</figure>

Creation and placment of ship.png would be the author's job. But it would live in harmony with the CAD version of the ship that was realized as ship.png. So the real source would be close at hand, in a repository with the rest of the source, and less likey to get lost over time.

[rbeezer]

Maybe if Will included his script here, we would all have a better idea of the requirements.

[whaynes]

Sure, here's the script.

I like it because I can organize the resource folder as I wish and keep the original source files right next to the images they produce.

The flatten() function finds and copies all the images to assets/images/. That way that I can reorganize the resource folder but the image path in the PreTeXt doesn't change. Folders that start with an underscore are ignored; good for material that's not yet ready for publication. If two images use the same filename one would get clobbered, so the script warns about that.

The copy_and_overwrite() function moves some other resources I need to their proper locations.

#!/usr/bin/env python3
import fnmatch
import os
import shutil

images = ['*.pdf', '*.png', '*.jpg', '*.svg', '*.ggb']
# these command line arguments are set by the makefile
resource_dir = "source/resources"  #(RESOURCES)
external_dir = "output/assets"  #(EXTERNAL)

def flatten(match_patterns, source, dest):
    print(f"\tFlattening {match_patterns} in {source} to {dest}.")

    def pattern_filter(path):
        for pattern in match_patterns:
            if fnmatch.fnmatch(path.lower(), pattern):
               return path

    # setup
    if os.path.exists(dest):
        shutil.rmtree(dest)
    os.makedirs(dest)

    # flatten source to dest
    for root, dirs, files in os.walk(source):
        for file in filter(pattern_filter, files):
            this_file = os.path.join(root, file)
            target = os.path.join(dest, file)
            if '/_' in root: continue  # Skip any files in directories starting with underscore.
            if os.path.exists(target):  # Warn if this filename has been seen before.
                raise RuntimeError(f"***Duplicate filename*** \"{file}\" shadows a file with same name somewhere else in the source folder.")
            shutil.copy2(this_file, target)

def copy_and_overwrite(source, dest):
    if not os.path.exists(source):
      print(f"\t***Resource Missing*** Can't find '{source}'.")
    else:
      print(f"\tCopying {source} to {dest}.")
      if os.path.exists(dest):
          shutil.rmtree(dest)
      shutil.copytree(source, dest)
      
def copy_and_preserve(source, dest):
    if not os.path.exists(source):
      print(f"\t***Resource Missing*** Can't find '{source}'.")
    else:
      print(f"\tCopying {source} to {dest}.")
      shutil.copytree(source, dest, dirs_exist_ok=True)

print("update_assets.py")
print()
print(f"\tWorking dir: {os.getcwd()}")
print(f"\tResource Directory: {resource_dir}")
print(f"\tExternal Directory: {external_dir}")
print()

flatten(images, resource_dir, external_dir + '/images')
copy_and_overwrite(resource_dir + '/_Numbas', external_dir + '/numbas')
copy_and_overwrite(resource_dir + '/common/_favicon', external_dir + '/favicon')
copy_and_overwrite(resource_dir + '/_css', external_dir + '/css')
copy_and_overwrite(resource_dir + '/_code', external_dir + '/code')
print()