background_tasks
Last updated: 2024-09-27 10:47:23.115925 File source: link on GitLab
background_tasks
Table of Contents
Specification
1. Description
The background_tasks
package is an internal package responsible for managing background jobs within DMS. It contains a scheduler that registers tasks and run them according to the schedule defined by the task definition.
proposed
Other packages that have their own background tasks register through this package:
Registration
The task itself, the arguments it needs
priority
event (time period or other event to trigger task)
Start , Stop, Resume
Algorithm that accounts for the event and priority of the task (not yet clear)
Monitor resource usage of tasks (not yet clear)
2. Structure and Organisation
Here is quick overview of the contents of this pacakge:
README: Current file which is aimed towards developers who wish to use and modify the package functionality.
init: This file initializes OpenTelemetry-based Zap logger.
scheduler: This file This file defines a background task scheduler that manages task execution based on triggers, priority, and retry policies.
task: This file contains background task structs and their properties.
trigger: This file defines various trigger types (PeriodicTrigger, EventTrigger, OneTimeTrigger) for background tasks, allowing execution based on time intervals, cron expressions, or external events
Files with *_test.go
naming convention contain unit tests of the functionality in corresponding file.
3. Class Diagram
Source
background_tasks class diagram
Rendered from source file
4. Functionality
NewScheduler
signature:
NewScheduler(maxRunningTasks int) *Scheduler
input:
maximum no of running tasks
output:
internal.background_tasks.Scheduler
NewScheduler
function creates a new scheduler which takes maxRunningTasks
argument to limit the maximum number of tasks to run at a time.
Scheduler methods
Scheduler
struct is the orchestrator that manages and runs the tasks. If the Scheduler
task queue is full, remaining tasks that are triggered will wait until there is a slot available in the scheduler.
It has the following methods:
AddTask
signature:
AddTask(task *Task) *Task
input:
internal.background_tasks.Task
output:
internal.background_tasks.Task
AddTask
registers a task to be run when triggered.
RemoveTask
signature:
RemoveTask(taskID int)
input:
identifier of the Task
output: None
RemoveTask
removes a task from the scheduler. Tasks with only OneTimeTrigger will be removed automatically once run.
Start
signature:
Start()
input: None
output: None
Start
starts the scheduler to monitor tasks.
Stop
signature:
Stop()
input: None
output: None
Stop
stops the scheduler.
runTask
signature:
runTask(taskID int)
input:
identifier of the Task
output: None
runTask
executes a task and manages its lifecycle and retry policy.
runTasks
signature:
runTasks()
input: None
output: None
runTasks
checks and runs tasks based on their triggers and priority.
runningTasksCount
signature:
runningTasksCount() int
input: None
output:
number of running tasks
runningTasksCount
returns the count of running tasks.
Trigger Interface
Its methods are explained below:
IsReady
signature:
IsReady() bool
input: None
output:
bool
IsReady
should return true if the task should be run.
Reset
signature:
Reset()
input: None
output: None
Reset
resets the trigger until the next event happens.
There are different implementations for the Trigger
interface.
PeriodicTrigger
: Defines a trigger based on a duration interval or a cron expression.EventTrigger
: Defines a trigger that is set by a trigger channel.OneTimeTrigger
: A trigger that is only triggered once after a set delay.
5. Data Types
internal.background_tasks.Scheduler
internal.background_tasks.RetryPolicy
internal.background_tasks.Execution
internal.background_tasks.Task
Task is a struct that defines a job. It includes the task's ID, Name, the function that is going to be run, the arguments for the function, the triggers that trigger the task to run, retry policy, etc.
internal.background_tasks.PeriodicTrigger
internal.background_tasks.EventTrigger
internal.background_tasks.OneTimeTrigger
6. Testing
Unit tests for each functionality are defined in files with *_test.go
naming convention.
7. Proposed Functionality / Requirements
List of issues
All issues that are related to the implementation of internal
package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.
8. References
Last updated