Model Overview
Domain Model
High-level Flow
Job Execution States
Application Models
Application
Contains basic metadata related to an application.
Table: application
Class Path: riberry.model.application.Application
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| document_id | NO | INTEGER | YES | Foreign key (document.id) | |
| name | NO | VARCHAR(64) | NO | The human-readable name of the application. | |
| internal_name | NO | VARCHAR(256) | NO | The internal name or secondary identifier of the application. | |
| A brief description of the application's purpose. | NO | VARCHAR(256) | YES | - | |
| type | NO | VARCHAR(64) | NO | The type of application. | |
| enabled | NO | BOOLEAN | YES | True | Whether or not this application and its instances are enabled (TODO). |
ApplicationInstance
An ApplicationInstance represents a running instance of an Application. Multiple ApplicationInstances are useful when we want to separate out different types of executions for the same Application.
An example of this is when we have an application which can support both long-running and short-running executions. We can spin up a separate instance with separate scheduling to ensure that the long-running jobs don't consume block the short-running jobs.
Table: app_instance
Class Path: riberry.model.application.ApplicationInstance
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| application_id | NO | INTEGER | NO | Foreign key (application.id) | |
| name | NO | VARCHAR(64) | NO | The human-readable name of the application. | |
| internal_name | NO | VARCHAR(256) | NO | The internal name or secondary identifier of the application instance. |
ApplicationInstanceSchedule
The ApplicationInstanceSchedule is a time-based schedule for dynamic parameters. It allows us to define parameters which change depending on the time of the day.
There are currently two core parameters: active and concurrency. These allow us to automatically
activate/de-activate and scale up/down our applications during different times of the day.
Any custom parameter can be defined and consumed by our Celery application as long as we have a custom parameter handler in place which can interpret the parameter's values.
Table: sched_app_instance
Class Path: riberry.model.application.ApplicationInstanceSchedule
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| instance_id | NO | INTEGER | NO | Foreign key (app_instance.id) | |
| days | NO | VARCHAR(27) | NO | Comma-separated list of specific days ("MON,WED"), or "*" for every day. | |
| start_time | NO | VARCHAR(8) | NO | 00:00:00 | The time when this schedule activates. |
| end_time | NO | VARCHAR(8) | NO | 23:59:59 | The time when this schedule de-activates. |
| timezone | NO | VARCHAR(128) | NO | UTC | The timezone of for the given start and end times. |
| parameter | NO | VARCHAR(32) | NO | The parameter which this schedule applies to. | |
| value | NO | VARCHAR(512) | NO | The value of the given parameter. | |
| priority | NO | INTEGER | NO | 64 | Priority of the schedule, where higher values mean higher priority. |
Heartbeat
The Heartbeat object is used by running ApplicationInstances to report back that they're alive. If an ApplicationInstance's last heartbeat was over 10 seconds ago, we assume that it is offline.
Table: heartbeat_app_instance
Class Path: riberry.model.application.Heartbeat
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| instance_id | NO | INTEGER | NO | Foreign key (app_instance.id) | |
| created | NO | DATETIME | YES | DATETIME (UTC) | The first heartbeat we received for the instance. |
| updated | NO | DATETIME | YES | DATETIME (UTC) | The last heartbeat we received for the instance. |
Auth Models
User
Table: users
Class Path: riberry.model.auth.User
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| username | NO | VARCHAR(48) | NO | - | |
| password | NO | VARCHAR(512) | YES | - | |
| auth_provider | NO | VARCHAR(32) | NO | default | - |
UserDetails
Table: user_details
Class Path: riberry.model.auth.UserDetails
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| user_id | NO | INTEGER | NO | Foreign key (users.id) | |
| first_name | NO | VARCHAR(64) | YES | - | |
| last_name | NO | VARCHAR(64) | YES | - | |
| display_name | NO | VARCHAR(128) | YES | - | |
| department | NO | VARCHAR(128) | YES | - | |
| NO | VARCHAR(128) | YES | - | ||
| updated | NO | DATETIME | YES | DATETIME (UTC) | - |
Group Models
Group
Table: groups
Class Path: riberry.model.group.Group
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| name | NO | VARCHAR(128) | NO | - | |
| display_name | NO | VARCHAR(128) | YES | - | |
| description | NO | VARCHAR(128) | YES | - |
ResourceGroupAssociation
Table: resource_group
Class Path: riberry.model.group.ResourceGroupAssociation
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| group_id | NO | INTEGER | NO | Foreign key (groups.id) | |
| resource_id | NO | INTEGER | NO | - | |
| resource_type | NO | VARCHAR(4) | NO | - |
Interface Models
ApplicationInterface
An ApplicationInterface defines a supported interface for an Application. An interface contains input file and value definitions which can be consumed.
ApplicationInterfaces are versioned, meaning you can have multiple interfaces with the same internal_name only if they have different versions.
Table: app_interface
Class Path: riberry.model.interface.ApplicationInterface
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| application_id | NO | INTEGER | NO | Foreign key (application.id) | |
| document_id | NO | INTEGER | YES | Foreign key (document.id) | |
| name | NO | VARCHAR(64) | NO | The human-readable name of the interface. | |
| internal_name | NO | VARCHAR(256) | NO | The internal name or secondary identifier of the interface. | |
| version | NO | INTEGER | NO | 1 | The version of the interface. |
| description | NO | VARCHAR(256) | YES | A brief description of the interface's purpose. |
Form
A Form is an implementation of an ApplicationInterface for a given ApplicationInstance.
All inputs a form accepts are defined by the ApplicationInterface it's linked to, and all Jobs created by the form belong to the linked ApplicationInstance.
Table: form
Class Path: riberry.model.interface.Form
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| instance_id | NO | INTEGER | NO | Foreign key (app_instance.id) | |
| interface_id | NO | INTEGER | NO | Foreign key (app_interface.id) | |
| enabled | NO | BOOLEAN | NO | True | Whether or not this form is enabled (TODO). |
FormSchedule
Table: sched_form
Class Path: riberry.model.interface.FormSchedule
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| form_id | NO | INTEGER | NO | Foreign key (form.id) | |
| start | NO | DATETIME | NO | - | |
| end | NO | DATETIME | NO | - |
InputFileDefinition
The InputFileDefinition object defines the properties of an input file.
Table: input_file_definition
Class Path: riberry.model.interface.InputFileDefinition
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| application_interface_id | NO | INTEGER | NO | Foreign key (app_interface.id) | |
| name | NO | VARCHAR(64) | NO | - | |
| internal_name | NO | VARCHAR(256) | NO | - | |
| description | NO | VARCHAR(128) | YES | - | |
| type | NO | VARCHAR(64) | NO | - | |
| accept | NO | VARCHAR(256) | YES | - | |
| required | NO | BOOLEAN | NO | True | - |
InputFileInstance
The InputFileInstance object contains data for a InputFileDefinition and is linked to a Job.
Table: input_file_instance
Class Path: riberry.model.interface.InputFileInstance
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| filename | NO | VARCHAR(512) | NO | - | |
| definition_id | NO | INTEGER | NO | Foreign key (input_file_definition.id) | |
| job_id | NO | INTEGER | NO | Foreign key (job.id) | |
| size | NO | INTEGER | NO | - | |
| binary | NO | BLOB | YES | - |
InputValueDefinition
The InputFileDefinition object defines the properties of an input value.
Table: input_value_definition
Class Path: riberry.model.interface.InputValueDefinition
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| application_interface_id | NO | INTEGER | NO | Foreign key (app_interface.id) | |
| name | NO | VARCHAR(64) | NO | - | |
| internal_name | NO | VARCHAR(256) | NO | - | |
| description | NO | VARCHAR(128) | YES | - | |
| type | NO | VARCHAR(64) | NO | - | |
| required | NO | BOOLEAN | NO | True | - |
| defaults | NO | BLOB | YES | - |
InputValueEnum
The InputValueEnum object defines a valid enumeration for a given InputValueInstance.
Table: input_value_enum
Class Path: riberry.model.interface.InputValueEnum
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| definition_id | NO | INTEGER | NO | Foreign key (input_value_definition.id) | |
| value | NO | BLOB | NO | - |
InputValueInstance
The InputValueInstance object contains data for a InputValueDefinition and is linked to a Job.
Table: input_value_instance
Class Path: riberry.model.interface.InputValueInstance
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| definition_id | NO | INTEGER | NO | Foreign key (input_value_definition.id) | |
| job_id | NO | INTEGER | NO | Foreign key (job.id) | |
| value | NO | BLOB | YES | - |
Job Models
Job
A Job is an object which represents a set of inputs provided to a form. A Job can have one or more JobExecutions which represent the execution of the linked ApplicationInterface on the linked ApplicationInstance for the given input stored against this Job.
Jobs are immutable. If we require a different set of values, we'll need to create a new Job.
Table: job
Class Path: riberry.model.job.Job
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| form_id | NO | INTEGER | NO | Foreign key (form.id) | |
| creator_id | NO | INTEGER | NO | Foreign key (users.id) | |
| name | NO | VARCHAR(64) | NO | The unique name of our job. | |
| created | NO | DATETIME | NO | DATETIME (UTC) | - |
JobExecution
A JobExecution represent a single execution of our Job.
Table: job_execution
Class Path: riberry.model.job.JobExecution
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| job_id | NO | INTEGER | NO | Foreign key (job.id) | |
| creator_id | NO | INTEGER | NO | Foreign key (users.id) | |
| task_id | NO | VARCHAR(36) | YES | The internal identifier of our job execution. This is usually the Celery root ID. | |
| status | NO | VARCHAR(24) | YES | RECEIVED | The current status of our job execution. |
| created | NO | DATETIME | NO | DATETIME (UTC) | - |
| started | NO | DATETIME | YES | - | |
| completed | NO | DATETIME | YES | - | |
| updated | NO | DATETIME | NO | DATETIME (UTC) | - |
| priority | NO | INTEGER | NO | 64 | The priority of this execution. This only applies to tasks in the RECEIVED state. |
JobExecutionArtifact
Table: job_artifact
Class Path: riberry.model.job.JobExecutionArtifact
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| job_execution_id | NO | INTEGER | NO | Foreign key (job_execution.id) | |
| stream_id | NO | INTEGER | YES | Foreign key (job_stream.id) | |
| name | NO | VARCHAR(128) | NO | - | |
| type | NO | VARCHAR(6) | NO | - | |
| category | NO | VARCHAR(128) | NO | Default | - |
| filename | NO | VARCHAR(512) | NO | - | |
| created | NO | DATETIME | NO | DATETIME (UTC) | - |
| size | NO | INTEGER | NO | - |
JobExecutionArtifactBinary
Table: job_artifact_binary
Class Path: riberry.model.job.JobExecutionArtifactBinary
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| binary | NO | BLOB | YES | - | |
| artifact_id | NO | INTEGER | NO | Foreign key (job_artifact.id) |
JobExecutionArtifactData
Table: job_artifact_data
Class Path: riberry.model.job.JobExecutionArtifactData
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| title | NO | VARCHAR(64) | NO | - | |
| description | NO | VARCHAR(512) | NO | - | |
| artifact_id | NO | INTEGER | NO | Foreign key (job_artifact.id) |
JobExecutionStream
Table: job_stream
Class Path: riberry.model.job.JobExecutionStream
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| job_execution_id | NO | INTEGER | NO | Foreign key (job_execution.id) | |
| task_id | NO | VARCHAR(36) | YES | - | |
| name | NO | VARCHAR(64) | YES | - | |
| status | NO | VARCHAR(24) | YES | QUEUED | - |
| created | NO | DATETIME | NO | DATETIME (UTC) | - |
| started | NO | DATETIME | YES | - | |
| completed | NO | DATETIME | YES | - | |
| updated | NO | DATETIME | NO | DATETIME (UTC) | - |
JobExecutionStreamStep
Table: job_stream_step
Class Path: riberry.model.job.JobExecutionStreamStep
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| stream_id | NO | INTEGER | NO | Foreign key (job_stream.id) | |
| task_id | NO | VARCHAR(36) | YES | - | |
| name | NO | VARCHAR(64) | YES | - | |
| status | NO | VARCHAR(24) | YES | RECEIVED | - |
| created | NO | DATETIME | NO | DATETIME (UTC) | - |
| started | NO | DATETIME | YES | - | |
| completed | NO | DATETIME | YES | - | |
| updated | NO | DATETIME | NO | DATETIME (UTC) | - |
JobSchedule
A JobSchedule object defines a schedule which will trigger an execution for the linked Job.
Note that a JobExecution will only be created for an ApplicationInstance which has a status of "online". Applications which are offline or inactive due to an ApplicationInstanceSchedule will not have JobExecutions created.
Table: sched_job
Class Path: riberry.model.job.JobSchedule
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| job_id | NO | INTEGER | NO | Foreign key (job.id) | |
| creator_id | NO | INTEGER | NO | Foreign key (users.id) | |
| enabled | NO | BOOLEAN | NO | True | Whether or not this schedule is active. |
| cron | NO | VARCHAR(24) | NO | The cron expression which defines our schedule. | |
| created | NO | DATETIME | NO | DATETIME (UTC) | The time our schedule was created. |
| last_run | NO | DATETIME | YES | The last time a job execution was created from our schedule. | |
| limit | NO | INTEGER | YES | 0 | The amount of valid runs for this schedule. |
| total_runs | NO | INTEGER | YES | 0 | The total amount of runs for this schedule. |
Misc Models
Document
Table: document
Class Path: riberry.model.misc.Document
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| type | NO | VARCHAR(24) | NO | markdown | - |
| content | NO | BLOB | NO | - |
Event
Table: event
Class Path: riberry.model.misc.Event
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| name | NO | VARCHAR(64) | NO | - | |
| time | NO | FLOAT | NO | - | |
| root_id | NO | VARCHAR(36) | NO | - | |
| task_id | NO | VARCHAR(36) | NO | - | |
| data | NO | VARCHAR(1024) | YES | - | |
| binary | NO | BLOB | YES | - |
Notification
Table: notification
Class Path: riberry.model.misc.Notification
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| type | NO | VARCHAR(7) | NO | NotificationType.info | - |
| message | NO | VARCHAR(128) | NO | - |
NotificationTarget
Table: notification_target
Class Path: riberry.model.misc.NotificationTarget
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| notification_id | NO | INTEGER | NO | Foreign key (notification.id) | |
| target | NO | VARCHAR(128) | NO | - | |
| target_id | NO | VARCHAR(128) | NO | - | |
| action | NO | VARCHAR(32) | YES | - |
UserNotification
Table: notification_user
Class Path: riberry.model.misc.UserNotification
Columns
| Column | Primary Key | Type | Nullable | Default | Comment |
|---|---|---|---|---|---|
| id | YES | INTEGER | NO | - | |
| user_id | NO | INTEGER | NO | Foreign key (users.id) | |
| notification_id | NO | INTEGER | NO | Foreign key (notification.id) | |
| created | NO | DATETIME | YES | DATETIME (UTC) | - |
| read | NO | BOOLEAN | NO | False | - |