Design Considerations

[zeryx]

Hey team, so this is a new repo; to let us ask some questions about it and figure out exactly what we want to do here, lets make a threaded discussion on this ticket.

Things we might want to talk about:

• CI / CD Automation
• Documentation
• User facing API design
• Algorithmia Client integration
• Examples
• Local debugging system

[zeryx]

Hey folks, pinging on this. Would ideally like feedback by mid week. Otherwise I'll be assuming this is functional.

[aslisabanci]

Hey James, I made a few edits on my local branch. If you can grant me access, I can push it and send you a PR for some minor edits. In the meantime, here's my initial thoughts:

• Load function must not have parameters: Not bringing this up as a blocker, but just to keep in mind: If a user wants to use a parametric model within the algorithm; ie. pass the model name or path to the apply function to dynamically tell the algorithm which model to load; then they wouldn't use ADK's load()

• assets folder is missing, for images you meant to put in README

• in ADK, we use the term state to represent the result/output of the user's load() function. Although I understand what we mean by that, "state" may not be the widely used/recognized word to describe this. To come up with an alternative, I'm thinking about the intended usage of this function. We're designing this function with the intention that users would load their model related files here; eg. the serialized model file; config file, checkpoints, word indexes, etc. So it's more like loaded configuration for the algorithm rather than a "state" in its canonical meaning. So I'm just opening this up to discussion to hear your thoughts.

• serve(input): We had brought up the naming of this function earlier. My alternative ideas are: process(input), run(input)

[zeryx]

1. This is a good point, especially for algorithms that could have a variety of operations that might change what's loaded into memory. We handle that in java/scala with an OOP solution by having essentially a hashmap as the load() functions return object type, so you can just update the object in that hashmap which will be preserved between runs. I'm thinking we could force the return type of load() to be a dict. The thought with load() is that it's a special function that only gets run once on container start. You could certainly call it yourself but that's not the intended purpose.

2. Yeah.. naming here is definitely something that we can play with, essentially all names are current placeholders, state is a bit nebulous; although I don't know if I'd prefer globals or something else - the difficulty is conveying that variables adjusted in say a hashmap will persist between calls.

3. I like process, run is a bit basic, I wonder since this will be something that could be iconic for us; if instead we had something like algorithm.initialize() or algorithm.execute() which are a bit stronger, but definitely open to changing the name.

[zeryx]

1. This is a good point, especially for algorithms that could have a variety of operations that might change what's loaded into memory. We handle that in java/scala with an OOP solution by having essentially a hashmap as the `load()` functions return object type, so you can just update the object in that hashmap which will be preserved between runs. I'm thinking we could force the return type of `load()` to be a `dict`. The thought with `load()` is that it's a special function that only gets run once on container start. You could certainly call it yourself but that's not the intended purpose.


1. Yeah.. naming here is definitely something that we can play with, essentially all names are current placeholders, state is a bit nebulous; although I don't know if I'd prefer `globals` or something else - the difficulty is conveying that variables adjusted in say a hashmap will persist between calls.

2. I like `process`, `run` is a bit basic, I wonder since this will be something that could be iconic for us; if instead we had something like `algorithm.initialize()` or `algorithm.execute()` which are a bit stronger, but definitely open to changing the name.

Thinking about this more, I'm actually a fan of .init(), but if you have other recommendations let me know. Still not sure about 2.

[aslisabanci]

This is a good point, especially for algorithms that could have a variety of operations that might change what's loaded into memory. We handle that in java/scala with an OOP solution by having essentially a hashmap as the load() functions return object type, so you can just update the object in that hashmap which will be preserved between runs. I'm thinking we could force the return type of load() to be a dict. The thought with load() is that it's a special function that only gets run once on container start. You could certainly call it yourself but that's not the intended purpose.

Yes, calling load() externally isn't the intended purpose. I was thinking about the case where the user wants to load different objects to memory for different algorithm requests; e.g. assume that you want to load different models depending on an input parameter in your algorithm, like currently you do apply(input, model_x) vs. apply(input, model_y) Then in the algorithm code you load model_x or model_y file to predict. So you wouldn't use ADK's load() for this purpose because the loaded objects are persisted between algorithm runs, in the container level. Just writing these to make sure my understanding is correct, let me know otherwise!

Yeah.. naming here is definitely something that we can play with, essentially all names are current placeholders, state is a bit nebulous; although I don't know if I'd prefer globals or something else - the difficulty is conveying that variables adjusted in say a hashmap will persist between calls.

I think globals is a good alternative to state. It conveys the meaning of a persisted & shared set of data, compared to what state means to me. When we say state, I think about something with transitions, rather than something static I guess. We can discuss if I'm alone in this or not.

I like process, run is a bit basic, I wonder since this will be something that could be iconic for us; if instead we had something like algorithm.initialize() or algorithm.execute() which are a bit stronger, but definitely open to changing the name.

.execute() and .init() sounds good to me too. Especially .init() gives me the impression that this is a function you'd call only once. Would you usually expect that you do something else after .init() or does it feel more like "init and go" to you guys?

[zeryx]

you won't call multiple apply functions - you won't even call that directly yourself, rather the init function will call it when a request comes in. I think globals is a reasonable name, what I'm going to do for both load event examples, is to have it return a dictionary. I don't know if we want to force it to be a dictionary, or just recommend it - but I think that's a good start.
Renaming to globals is reasonable, going to do that now.

[aslisabanci]

No I wasn't saying we'd call multiple apply() functions. I gave an example of two different apply() function calls a user can do, as of now - without ADK. So what I meant was; assume that you have an apply function that takes two parameters like apply(input, model_name). When you send two different requests to your algorithm as:

• algo.pipe("input1", "model_x") would in turn call apply("input1", "model_x")
• algo.pipe("input2", "model_y") would in turn call apply("input2", "model_y") right?
  So I was giving an example in our current structure; to explain the scenario in my mind.