Scheduler

Introduction

The Panel platform scheduler consists of a windows service for running schedules, an interactive command prompt that can run in service mode for diagnostics, a schedule editor in the Identity Panel settings section, and dashboard modules for running and monitoring schedules.

The scheduler allows creation of multiple schedules. A schedule is configured with the days and weeks that it runs, the times it starts, and for how long it repeats. A schedule consists of a sequence of steps. Which steps are available depends on which Panel platform modules are installed and enabled.

Scheduler Features

  • Run multiple schedules simultaneously
  • Invoke schedules manually from Web UI
  • Cancel pending schedules/steps from Web UI
  • Only run on particular days of the month
  • Steps from a single schedule can execute on different servers and stay synchronized
  • Test the state of the environment before running a step
  • Skip steps based on the result of running a step
  • Trigger multiple steps simultaneously if the steps don't conflict with each other
  • Fault tolerance to continue schedule execution in event of server failure
  • Track results of schedule operations over time
  • Extensibility for creating custom step types

Schedule Concurrency

The Panel Scheduler can run an arbitrary number of schedules against an arbitrary number of servers simultaneously. It is also able to trigger combinations of steps simultaneously. To prevent concurrency errors, each step has locking rules to provide for step isolation.

If incompatible steps would occur simultaneously, whichever step is lower in the list will be postponed until the previous step terminates. If incompatible steps are in separate simultaneously occurring schedules, then whichever schedule was started first, will generally have it's next steps dispatched first. All schedule step dispatch, concurrency, and precedence is coordinated by the web application, which allows locking to be respected in the case of server failure, and manual schedule execution.

See Step Types for documentation on the exclusivity rules for each type of step.

Running Schedules

Identity Panel includes a dashboard module which can be used to start and manage running schedules. To configure this, navigate to the Dashboard settings and create a new Dashboard or modify an existing one. The dashboard should be associated to an administrative security role. Add a new module to the dashboard choosing the "View and Run Schedules" type.
Schedule Dashboard Module

Once the dashboard has been modified, navigate to the Identity Panel home page. To start a schedule ad-hoc, select it's name from the drop-down, and press the Run button.
Schedule Module

If a schedule is in the queue or has running steps it will appear as a strip in the dashboard module. If you expand the strip you can see the names of steps, whether they are Pending or Running, and if Running, which server they are executing on. If you pause a schedule, all non-running steps will be placed in the Paused state. You then have the option to resume the whole schedule, or to step forward manually one step at a time.

Viewing Schedule Progress

Identity Panel 3.0 introduces a new dashboard console module which can be used to display aggregated output from Panel Service, including schedule and workflow progress. Nearly all StdOut and StdErr output from PanelTool and PanelService get written to the Identity Panel log collection. To view the most recent 24 hours of output in realtime, add the "View Output from Panel Tools" dashboard module. After adding the module you must select which servers you want to show output from.

images/tools_dashboard_settings.png|Tools Dashboard Settings

Tools Dashboard

Using the Schedule Editor

Navigate to the Settings tab and click on the Schedules section. This interface can be used to add, remove, and modify schedules.

Editing Controls

The schedule editor has global controls and per schedule controls.

Schedule Global Controls

Global Controls

  • Use the Name and Filter text boxes to find specific schedules if you have a large number of schedules created.
  • Use the arrow in a circle icons to upload and download schedule settings as JSON files.
  • Pressing the Save button will validate schedule settings, and if no errors are found, will save schedule settings.
  • The Enabled checkbox under settings will enable or disable scheduler globally.
  • The Disable History checkbox will disable automatic data collection of run history information.
  • The "New" buttons adds schedules and steps

Schedule Editor

Individual Schedule Controls

  • Schedules can be expanded and collapsed by clicking the arrow next to the schedule name.
  • Use the Enabled checkbox to enable and disable individual schedules. There is also a global Enabled checkbox for the whole scheduler.
  • The "X" buttons deletes schedules and steps. When you remove a schedule it is permanently and immediately removed from Identity Panel.
  • Drag and drop to re-order schedules and steps.
  • The up and down arrows inside a circle will upload and download individual schedules as JSON files.
  • Selecting "Disable data collection for Time Traveler" will completely disable data collection. If you solely intend to use Panel platform for scheduling, this can modestly speed up schedule execution; however, you will be unable to access historical data.

Schedule Settings

All schedules are required to have a unique name. This allows them to be identified and selected in the UI.

When a schedule is created the start time list can be left empty. Schedules without a start time are available to run manually from the settings interface. This can be useful for queueing up a series of long running tasks to run unattended.

To run in the Scheduler service, a schedule must have at least one day-of-the-week checkbox selected, and at least one week-of-the-month checkbox selected, and must have a start time. If desired, a schedule can be configured to repeat at an interval until a given end time. Validation permits you to specify just a Start time, but if you specify a repeat interval, then you must provide values for all three time fields.

If a schedule is specified to run on a repeat interval, and the schedule is still running when the interval expires, then that iteration will be skipped. This prevents a rapidly repeating schedule from building up a long queue that consumes resources and potentially interferes with other schedules.

The week of the month is calculated based on seven-day intervals starting from the first day of the month. If the month starts on a Wednesday, then each week runs from Wednesday to Tuesday. This makes it possible, for example, to only run a schedule on the first Monday of the month, or the third Friday.

Day boundaries for schedules are selected based on the chosen time zone.

To run schedules concurrently, simply specify overlapping start times.

To add steps to a schedule, first select the type of step from the drop-down list, then press the Add button.

Step Settings

The list of available steps depends on which modules have been installed. Steps are loaded from the Extensions folder, and the source code for SoftwareIDM step implementations is available. New steps can be added by implementing the schedule step interfaces. For details see Panel Extensions.

Each schedule step has some settings that are general to all steps, and some that are step specific. Step specific settings are documented under Step Types. General settings are:

ASync

To run steps concurrently, select the Async checkbox in the step strip.

If steps have ASync selected, they will be started simultaneously. At the first step that is not-selected ASync, the scheduler will wait for all running ASync steps to complete. If two steps are mutually incompatible and both are selected ASync, then whichever step is placed first in the list will run to completion before the next step starts.

Preferred Schedulers

The preferred scheduler drop-down list is populated based on the servers that have Panel Tools installed. If this drop-down is left blank, the step will run on whichever Panel Service queries for pending steps first when the step is queued up to run. Otherwise, the scheduler will wait up to the Inactive Threshold (typically ten minutes), for the preferred service to query. Panel Service will continue to poll for available steps even while other steps are running.

If the inactive threshold is passed, the scheduler will check the value of the "Only Preferred" checkbox. If this is not selected, the step will be dispatched to a different Panel Service if one is available. Otherwise the step will wait indefinitely until one of the preferred services is available.

Condition Rule

After a step has been dispatched to the Panel Service, but before it executes, the service uses the Rule Engine to evaluate the condition rule. The "context" for the rule is the settings for the step, but typically condition rules will not require Object Property lookups. For example, a condition rule might check the number of pending export deletes on an MA prior to executing an export run-profile step. If the condition rule is not defined, the step will run. Otherwise the step will only run if the condition rule is true. If the step does not run, the history counter will be set to Skipped.

Condition Rule Pauses Schedule

If the Condition Rule Pauses Schedule checkbox is selected, the entire schedule will be placed in a suspended state if the condition rule returns false. This can be used to detect critical conditions in schedule execution. It is also possible to use the workflow engine to send an alert when a schedule enters a paused state.

Skip Rule

When a step completes, it returns a History Record representing the results of what it did. The type of history record depends on the step. For example, if the step executed a PowerShell script or a program, it will return a "Program History Record". If the step ran an MA, it will return an "MA Run Record". This history record is then used as the "context" for the Skip Rule. The Skip Rule is evaluated by the Panel Service immediately after the step completes. The result is coerced to a number, which determines how many queued steps to skip from that particular schedule. The Skip Rule is evaluated regardless of whether or not the step was run asynchronously, but steps that are already in-progress will not be cancelled.

Schedule Step Types

Panel platform includes a variety of step types that can be added to a schedule. As new modules become available this list will expand.

For additional schedule steps see Provider documentation.

Data Retention

The Data Retention step applies a data retention policy to all data stored in the Panel platform database.

Data Retention

History

Returns a simple History Record with { RecordOf = "Data Retention", Result="success|error message"} .

Concurrency

Data Retention may run simultaneously with anything but another Data Retention step.

Settings

  • Quantity: Required, Amount of time to retain history for
  • Unit: Required, Time unit for quantity (typically Years should be selected)

Run PowerShell

The Run PowerShell step executes a PowerShell script, either against the local server or using PowerShell remoting. The majority of the settings in the Run PowerShell step are ignored unless the "Use PS Remoting" checkbox is selected. The Run PowerShell step has many options to allowed detailed control over establishing a remoting session. Note: commands that require an interactive shell, such as Write-Host will fail. Content written to the Output, Warning, or Error streams is logged by the History Record.

Run PowerShell

History

Returns a Program History Record, with RecordOf set to "PowerShell", and Argument set to the Name parameter. This History Record is saved with each of the first ten kilobytes of the Output and Error streams.

Concurrency

The PowerShell step may run simultaneously with anything but a PowerShell step of the same Name.

Settings

  • Name: Required, used for run history identification, and for concurrency locking.
  • Use PS Remoting: decides whether to examine connection settings or simply run locally.
  • Script: Required, the PowerShell script to run.
  • Host: Host name if using PS Remoting
  • User: User name for PS Remoting
  • Password: Password for PS Remoting login. This may be overridden by the Panel:Password app setting.
  • Auth Mechanism: PowerShell Auth Mechanism enumeration value.
  • URI: sub-path combined with Host and protocol to create the connection URL.
  • Schema URI: URI for PowerShell command schema. The default PowerShell schema is: http://schemas.microsoft.com/powershell/Microsoft.PowerShell and the default Exchange 2013 schema is: http://schemas.microsoft.com/powershell/Microsoft.Exchange
  • Use SSL: whether to use http: or https: for remoting
  • Port: Default PowerShell http: port is 5985. Default PowerShell https: port is 5986.
  • Operation Timeout Seconds: Number of seconds to wait for response from a command before timing out. Defaults to 240 seconds (4 minutes).
  • Max Redirect Count: Number of times to allow redirection from the PowerShell URL. Defaults to 3 redirects.

Run Program

The run program step executes an arbitrary script or program.

Run Program

History

Returns a Program History Record with RecordOf set to the Program, and Argument set to the Args string. This History Record is saved with each of the first ten kilobytes of the Output and Error streams.

Concurrency

The program step may run simultaneously with anything but a Run Program step that has the same Panel Service, Program, and Args.

Settings

  • Program Path: Required, can be a full path or the name of the program if the program is on the PATH environment variable.
  • Args: Optional, arguments string to pass to the program.

Running Schedules

Under normal conditions schedules will only execute if one or more instances of SoftwareIDM Panel Service are running.

Occasionally it is desirable for troubleshooting to run schedules from the command line and view output immediately. To do this, start PanelTool, and choose the Service Mode option from the menu. This will cause PanelTool to stop the Panel Service if it is running, and perform all service operations until you press 'Esc' or 'spacebar'.

Copyright © SoftwareIDM

Table of Contents