Skip to content

Latest commit

 

History

History
1925 lines (933 loc) · 49.1 KB

File metadata and controls

1925 lines (933 loc) · 49.1 KB

Protocol Documentation

Table of Contents

Top

o2control.proto

AddRepoReply

Field Type Label Description
newDefaultRevision string
info string
timestamp int64 timestamp of when this object was sent in unix milliseconds

AddRepoRequest

Field Type Label Description
name string
defaultRevision string

ChannelInfo

Field Type Label Description
name string
type string
target string

CleanupTasksReply

Field Type Label Description
killedTasks ShortTaskInfo repeated
runningTasks ShortTaskInfo repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

CleanupTasksRequest

Field Type Label Description
taskIds string repeated

CommandInfo

Field Type Label Description
env string repeated
shell bool
value string
arguments string repeated
user string

ControlEnvironmentReply

Field Type Label Description
id string
state string
currentRunNumber uint32
startOfTransition int64 All times are in milliseconds
endOfTransition int64
transitionDuration int64
timestamp int64 timestamp of when this object was sent in unix milliseconds

ControlEnvironmentRequest

Field Type Label Description
id string
type ControlEnvironmentRequest.Optype
requestUser common.User

DestroyEnvironmentReply

Field Type Label Description
cleanupTasksReply CleanupTasksReply
timestamp int64 timestamp of when this object was sent in unix milliseconds

DestroyEnvironmentRequest

Field Type Label Description
id string
keepTasks bool
allowInRunningState bool
force bool
requestUser common.User

Empty

EnvironmentInfo

Field Type Label Description
id string
createdWhen int64 msec
state string
tasks ShortTaskInfo repeated
rootRole string
currentRunNumber uint32
defaults EnvironmentInfo.DefaultsEntry repeated
vars EnvironmentInfo.VarsEntry repeated
userVars EnvironmentInfo.UserVarsEntry repeated
numberOfFlps int32
includedDetectors string repeated
description string
numberOfHosts int32
integratedServicesData EnvironmentInfo.IntegratedServicesDataEntry repeated
numberOfTasks int32
currentTransition string
numberOfActiveTasks int32
numberOfInactiveTasks int32

EnvironmentInfo.DefaultsEntry

Field Type Label Description
key string
value string

EnvironmentInfo.IntegratedServicesDataEntry

Field Type Label Description
key string
value string

EnvironmentInfo.UserVarsEntry

Field Type Label Description
key string
value string

EnvironmentInfo.VarsEntry

Field Type Label Description
key string
value string

EnvironmentOperation

Field Type Label Description
type EnvironmentOperation.Optype
roleName string

GetActiveDetectorsReply

Field Type Label Description
detectors string repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetAvailableDetectorsReply

Field Type Label Description
detectors string repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetEnvironmentPropertiesReply

Field Type Label Description
properties GetEnvironmentPropertiesReply.PropertiesEntry repeated

GetEnvironmentPropertiesReply.PropertiesEntry

Field Type Label Description
key string
value string

GetEnvironmentPropertiesRequest

Field Type Label Description
id string
queries string repeated If len(queries) == 0, we return an empty map. To retrieve all KVs, use query '*'
excludeGlobals bool

GetEnvironmentReply

Field Type Label Description
environment EnvironmentInfo
workflow RoleInfo
public bool
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetEnvironmentRequest

Field Type Label Description
id string
showWorkflowTree bool

GetEnvironmentsReply

Field Type Label Description
frameworkId string
environments EnvironmentInfo repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetEnvironmentsRequest

Environment //////////////////////////////////////

Field Type Label Description
showAll bool
showTaskInfos bool
showDetailedIntegratedServices bool integratedServices are returned everytime, setting this flag gives detailed report

GetFrameworkInfoReply

Field Type Label Description
frameworkId string
environmentsCount int32
tasksCount int32
state string
hostsCount int32
instanceName string
version Version
configurationEndpoint string
detectorsInInstance string repeated
activeDetectors string repeated
availableDetectors string repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetFrameworkInfoRequest

Framework //////////////////////////////////////

GetRolesReply

Field Type Label Description
roles RoleInfo repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetRolesRequest

Roles //////////////////////////////////////

Field Type Label Description
envId string
pathSpec string

GetTaskReply

Field Type Label Description
task TaskInfo
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetTaskRequest

Field Type Label Description
taskId string

GetTasksReply

Field Type Label Description
tasks ShortTaskInfo repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetTasksRequest

GetWorkflowTemplatesReply

Field Type Label Description
workflowTemplates WorkflowTemplateInfo repeated
timestamp int64 timestamp of when this object was sent in unix milliseconds

GetWorkflowTemplatesRequest

Field Type Label Description
repoPattern string
revisionPattern string
allBranches bool
allTags bool
allWorkflows bool

IntegratedServiceInfo

Field Type Label Description
name string user-visible service name, e.g. "DD scheduler"
enabled bool
endpoint string
connectionState string allowed values: READY, CONNECTING, TRANSIENT_FAILURE, IDLE, SHUTDOWN
data string always a JSON payload with a map<string, string> inside.

ListIntegratedServicesReply

Field Type Label Description
services ListIntegratedServicesReply.ServicesEntry repeated keys are IDs (e.g. "ddsched"), the service name should be displayed to users instead
timestamp int64 timestamp of when this object was sent in unix milliseconds

ListIntegratedServicesReply.ServicesEntry

Field Type Label Description
key string
value IntegratedServiceInfo

ListReposReply

Field Type Label Description
repos RepoInfo repeated
globalDefaultRevision string
timestamp int64 timestamp of when this object was sent in unix milliseconds

ListReposRequest

Field Type Label Description
getRevisions bool

ModifyEnvironmentReply

Field Type Label Description
failedOperations EnvironmentOperation repeated
id string
state string

ModifyEnvironmentRequest

Field Type Label Description
id string
operations EnvironmentOperation repeated
reconfigureAll bool

NewAutoEnvironmentReply

Field Type Label Description
timestamp int64 timestamp of when this object was sent in unix milliseconds

NewAutoEnvironmentRequest

Field Type Label Description
workflowTemplate string
vars NewAutoEnvironmentRequest.VarsEntry repeated
id string
requestUser common.User

NewAutoEnvironmentRequest.VarsEntry

Field Type Label Description
key string
value string

NewEnvironmentReply

Field Type Label Description
environment EnvironmentInfo
public bool
timestamp int64 timestamp of when this object was sent in unix milliseconds

NewEnvironmentRequest

Field Type Label Description
workflowTemplate string
vars NewEnvironmentRequest.VarsEntry repeated
public bool
autoTransition bool
requestUser common.User

NewEnvironmentRequest.VarsEntry

Field Type Label Description
key string
value string

RefreshReposRequest

Field Type Label Description
index int32

RemoveRepoReply

Field Type Label Description
newDefaultRepo string
timestamp int64 timestamp of when this object was sent in unix milliseconds

RemoveRepoRequest

Field Type Label Description
index int32

RepoInfo

Field Type Label Description
name string
default bool
defaultRevision string
revisions string repeated

RoleInfo

Field Type Label Description
name string
status string
state string
fullPath string
taskIds string repeated
roles RoleInfo repeated
defaults RoleInfo.DefaultsEntry repeated
vars RoleInfo.VarsEntry repeated
userVars RoleInfo.UserVarsEntry repeated
consolidatedStack RoleInfo.ConsolidatedStackEntry repeated
description string

RoleInfo.ConsolidatedStackEntry

Field Type Label Description
key string
value string

RoleInfo.DefaultsEntry

Field Type Label Description
key string
value string

RoleInfo.UserVarsEntry

Field Type Label Description
key string
value string

RoleInfo.VarsEntry

Field Type Label Description
key string
value string

SetDefaultRepoRequest

Field Type Label Description
index int32

SetEnvironmentPropertiesReply

SetEnvironmentPropertiesRequest

Environment, GET/SET properties //////////////////////////////////////

Field Type Label Description
id string
properties SetEnvironmentPropertiesRequest.PropertiesEntry repeated If properties == nil, the core sets nothing and reply ok

SetEnvironmentPropertiesRequest.PropertiesEntry

Field Type Label Description
key string
value string

SetGlobalDefaultRevisionRequest

Field Type Label Description
revision string

SetRepoDefaultRevisionReply

Field Type Label Description
info string
timestamp int64 timestamp of when this object was sent in unix milliseconds

SetRepoDefaultRevisionRequest

Field Type Label Description
index int32
revision string

ShortTaskInfo

Tasks //////////////////////////////////////

Field Type Label Description
name string
locked bool
taskId string
status string
state string
className string
deploymentInfo TaskDeploymentInfo
pid string
sandboxStdout string
claimable bool
critical bool

SubscribeRequest

Field Type Label Description
id string

TaskDeploymentInfo

Field Type Label Description
hostname string
agentId string
offerId string
executorId string

TaskInfo

Field Type Label Description
shortInfo ShortTaskInfo
inboundChannels ChannelInfo repeated
outboundChannels ChannelInfo repeated
commandInfo CommandInfo
taskPath string
envId string
properties TaskInfo.PropertiesEntry repeated

TaskInfo.PropertiesEntry

Field Type Label Description
key string
value string

TeardownReply

TeardownRequest

Not implemented yet

Field Type Label Description
reason string

VarSpecMessage

Field Type Label Description
defaultValue string
type VarSpecMessage.Type
label string
description string
widget VarSpecMessage.UiWidget
panel string hint for the UI on where to put or group the given variable input
allowedValues string repeated list of offered values from which to choose (only for some UiWidgets)
index int32
visibleIf string JS expression that evaluates to bool
enabledIf string JS expression that evaluates to bool
rows uint32 this field is used only if widget == editBox

Version

Field Type Label Description
major int32
minor int32
patch int32
build string
productName string
versionStr string

WorkflowTemplateInfo

Field Type Label Description
repo string
template string
revision string
varSpecMap WorkflowTemplateInfo.VarSpecMapEntry repeated
description string

WorkflowTemplateInfo.VarSpecMapEntry

Field Type Label Description
key string
value VarSpecMessage

ControlEnvironmentRequest.Optype

Name Number Description
NOOP 0
START_ACTIVITY 1
STOP_ACTIVITY 2
CONFIGURE 3
RESET 4
GO_ERROR 5
DEPLOY 6

EnvironmentOperation.Optype

Name Number Description
NOOP 0
REMOVE_ROLE 3
ADD_ROLE 4

VarSpecMessage.Type

Name Number Description
string 0
number 1
bool 2
list 3
map 4

VarSpecMessage.UiWidget

Name Number Description
editBox 0 plain string input line, can accept types number (like a spinBox) and string
slider 1 input widget exclusively for numbers, range allowedValues[0]-[1]
listBox 2 displays a list of items, can accept types number, string or list; if number/string ==> single selection, otherwise multiple selection allowed
dropDownBox 3
comboBox 4
radioButtonBox 5
checkBox 6

Control

The Control service is the main interface to AliECS

Method Name Request Type Response Type Description
GetFrameworkInfo GetFrameworkInfoRequest GetFrameworkInfoReply
GetEnvironments GetEnvironmentsRequest GetEnvironmentsReply
NewAutoEnvironment NewAutoEnvironmentRequest NewAutoEnvironmentReply Creates a new environment which automatically follows one STANDBY->RUNNING->DONE cycle in the state machine. It returns only once the environment reaches the CONFIGURED state or upon any earlier failure.
NewEnvironment NewEnvironmentRequest NewEnvironmentReply Creates a new environment. It returns only once the environment reaches the CONFIGURED state or upon any earlier failure.
GetEnvironment GetEnvironmentRequest GetEnvironmentReply
ControlEnvironment ControlEnvironmentRequest ControlEnvironmentReply
DestroyEnvironment DestroyEnvironmentRequest DestroyEnvironmentReply
GetActiveDetectors Empty GetActiveDetectorsReply
GetAvailableDetectors Empty GetAvailableDetectorsReply
NewEnvironmentAsync NewEnvironmentRequest NewEnvironmentReply Creates a new environment. It returns once an environment ID is created and continues the creation asynchronously to the call. The environment will be listed in GetEnvironments() only once the workflow is loaded and deployment starts.
GetTasks GetTasksRequest GetTasksReply
GetTask GetTaskRequest GetTaskReply
CleanupTasks CleanupTasksRequest CleanupTasksReply
GetRoles GetRolesRequest GetRolesReply
GetWorkflowTemplates GetWorkflowTemplatesRequest GetWorkflowTemplatesReply
ListRepos ListReposRequest ListReposReply
AddRepo AddRepoRequest AddRepoReply
RemoveRepo RemoveRepoRequest RemoveRepoReply
RefreshRepos RefreshReposRequest Empty
SetDefaultRepo SetDefaultRepoRequest Empty
SetGlobalDefaultRevision SetGlobalDefaultRevisionRequest Empty
SetRepoDefaultRevision SetRepoDefaultRevisionRequest SetRepoDefaultRevisionReply
Subscribe SubscribeRequest .events.Event stream
GetIntegratedServices Empty ListIntegratedServicesReply
Teardown TeardownRequest TeardownReply Reserved and not implemented:
ModifyEnvironment ModifyEnvironmentRequest ModifyEnvironmentReply

Top

protos/events.proto

Ev_BeamModeEvent

Beam mode changes are propagated as Kafka events and to be sent by the BKP-LHC-Client on a dedicated topic e.g. dip.lhc.beam_mode

Field Type Label Description
timestamp int64 milliseconds since epoch when the beam mode change happened
beamInfo common.BeamInfo

Ev_CallEvent

Field Type Label Description
func string name of the function being called, within the workflow template context
callStatus OpStatus progress or success/failure state of the call
return string return value of the function
traits Traits
output string any additional output of the function
error string error value, if returned
environmentId string
path string path to the parent callRole of this call within the environment

Ev_EnvironmentEvent

Field Type Label Description
environmentId string
state string
runNumber uint32 only when the environment is in the running state
error string
message string any additional message concerning the current state or transition
transition string
transitionStep string
transitionStatus OpStatus
vars Ev_EnvironmentEvent.VarsEntry repeated consolidated environment variables at the root role of the environment
lastRequestUser common.User
workflowTemplateInfo common.WorkflowTemplateInfo

Ev_EnvironmentEvent.VarsEntry

Field Type Label Description
key string
value string

Ev_IntegratedServiceEvent

Field Type Label Description
name string name of the context, usually the path of the callRole that calls a given integrated service function e.g. readout-dataflow.dd-scheduler.terminate
error string error message, if any
operationName string name of the operation, usually the name of the integrated service function being called e.g. ddsched.PartitionTerminate()"
operationStatus OpStatus progress or success/failure state of the operation
operationStep string if the operation has substeps, this is the name of the current substep, like an API call or polling phase
operationStepStatus OpStatus progress or success/failure state of the current substep
environmentId string
payload string any additional payload, depending on the integrated service; there is no schema, it can even be the raw return structure of a remote API call

Ev_MetaEvent_CoreStart

Field Type Label Description
frameworkId string

Ev_MetaEvent_FrameworkEvent

Field Type Label Description
frameworkId string
message string

Ev_MetaEvent_MesosHeartbeat

Ev_RoleEvent

Field Type Label Description
name string role name
status string posible values: ACTIVE/INACTIVE/PARTIAL/UNDEFINED/UNDEPLOYABLE as defined in status.go. Derived from the state of child tasks, calls or other roles
state string state machine state for this role
rolePath string path to this role within the environment
environmentId string

Ev_RunEvent

Field Type Label Description
environmentId string
runNumber uint32
state string
error string
transition string
transitionStatus OpStatus
lastRequestUser common.User

Ev_TaskEvent

Field Type Label Description
name string task name, based on the name of the task class
taskid string task id, unique
state string state machine state for this task
status string posible values: ACTIVE/INACTIVE/PARTIAL/UNDEFINED/UNDEPLOYABLE as defined in status.go.
hostname string
className string name of the task class from which this task was spawned
traits Traits
environmentId string
path string path to the parent taskRole of this task within the environment

Event

Field Type Label Description
timestamp int64
timestampNano int64
environmentEvent Ev_EnvironmentEvent Events produced by AliECS
taskEvent Ev_TaskEvent
roleEvent Ev_RoleEvent
callEvent Ev_CallEvent
integratedServiceEvent Ev_IntegratedServiceEvent
runEvent Ev_RunEvent
frameworkEvent Ev_MetaEvent_FrameworkEvent Meta events produced by AliECS or its components
mesosHeartbeatEvent Ev_MetaEvent_MesosHeartbeat
coreStartEvent Ev_MetaEvent_CoreStart
beamModeEvent Ev_BeamModeEvent Events produced by other systems, but natively supported and defined by AliECS

Traits

Field Type Label Description
trigger string
await string
timeout string
critical bool

OpStatus

Name Number Description
NULL 0
STARTED 1
ONGOING 2
DONE_OK 3
DONE_ERROR 4
DONE_TIMEOUT 5

Top

protos/common.proto

BeamInfo

Beam information at a specific point in time (e.g. start or end of stable beams)

Field Type Label Description
stableBeamsStart int64 milliseconds since epoch when stable beams started
stableBeamsEnd int64 milliseconds since epoch when stable beams ended
fillNumber int32 LHC fill number
fillingSchemeName string LHC filling scheme name e.g. 25ns_2460b_2448_2089_2227_144bpi_20inj
beam1Energy float in GeV
beam2Energy float in GeV
beamType string e.g. PROTON-PROTON, O8-O8, Pb-Pb, p-Pb, Pb-p
beamMode BeamMode

User

Field Type Label Description
externalId int32 optional The unique CERN identifier of this user.
id int32 optional The unique identifier of this entity.
name string Name of the user.

WorkflowTemplateInfo

Field Type Label Description
name string
description string
path string
public bool whether the environment is public or not

BeamMode

Beam modes as defined and sent by LHC DIP client plus:

Name Number Description
UNKNOWN 0 virtual type
SETUP 1
ABORT 2
INJECTION_PROBE_BEAM 3
INJECTION_SETUP_BEAM 4
INJECTION_PHYSICS_BEAM 5
PREPARE_RAMP 6
RAMP 7
FLAT_TOP 8
SQUEEZE 9
ADJUST 10
STABLE_BEAMS 11
LOST_BEAMS 12 virtual type
UNSTABLE_BEAMS 13
BEAM_DUMP_WARNING 14
BEAM_DUMP 15
RAMP_DOWN 16
CYCLING 17
RECOVERY 18
INJECT_AND_DUMP 19
CIRCULATE_AND_DUMP 20
NO_BEAM 21

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)