Latest version
Close
Close
Firetask for Mac is a task management app that lets you enter tasks in various categories and under different projects. You can rank each task in terms of. Firetask 3.8.3 – Innovative task management solution. Firetask uniquely combines the advantages of classical priority-and-due-date-based task management with GTD. Stay focused and on top of your commitments – Firetask’s “Today” view shows all relevant due and next tasks in a single, clear overview. Firetask 5.8.5 is recommended for all users and contains minor bug fixes and usability improvements. Verdict: It could easily be argued that Firetask is a little on the expensive side, but what it does, it does wonderfully – you just need to decide whether you are willing to pay the premium. Firetask was added by geraldaquila in Oct 2010 and the latest update was made in Mar 2020. The list of alternatives was updated Sep 2020. It's possible to update the information on Firetask or report it as discontinued, duplicated or spam. Firetask for Mac is a task management app that lets you enter tasks in various categories and under different projects. You can rank each task in terms of.
Released:
Run FireWorks workflows in Google Cloud
Project description
Runs FireWorks workflows onGoogle Cloud Platform computing services (GCP).
See the repo Borealis and thePyPI page borealis-fireworks.
What is it?
FireWorks is an open-sourceworkflow management system, that is, it drives interdependent computing tasks.Among the many workflow systems available,FireWorks is exceptionally straightforward, lightweight, andadaptable. It's well tested and supported. The only shared services it needs area MongoDB server (acting as the workflow 'LaunchPad') and a file store.
Borealis lets you spin up as many temporary worker machines as you want inthe Google Cloud Platform to run yourworkflow. That means pay-per-use and no contention between workflows.
How does Borealis support workflows on Google Cloud Platform?
TL;DR: Spin up worker machines when you need them, deploy your task code tothe workers in Docker Images, and store the data in Google Cloud Storage insteadof NFS.
Figure 1. Blue hexagons represent Google Cloud Platform services.The LaunchPad is shown in a blue hexagon because it's implemented as aMongoDB server that you can run on Compute Engine.
Worker VMs: As a cloud computing platform, Google ComputeEngine (GCE) has a vast number of machinesavailable. You can spin up lots of GCE 'instances' (also called Virtual Machinesor VMs) to run your workflow, change your code, re-run some tasks, then let theworkers time out and shut down. Google will charge you based on usage andthere's no resource contention with your teammates.
Borealis provides the
ComputeEngine
class and its command line wrapper gce
to create, tweak, and delete groups of worker VMs.Borealis provides the
fireworker
Python script to run as the top level programof each worker. It calls FireWorks' rlaunch
feature.You can run these Fireworkers on and off GCE as long as they can connect toyour MongoDB server and to the data store for their input and output files.
Docker: You need to deploy your payload task code to those GCE VMs. Itmight be Python source code and its runtime environment, e.g. Python 3.8,Python pip packages, Linux apt packages, compiledCython code, data files, and environment variable settings. A GCE VM starts upfrom a GCE Disk Image which could have all that preinstalled (with orwithout the Python source code) but it'd be hard to keep it up to date andhard to keep track of how to reproduce it.
This is what Docker Images are designed for. You maintain a
Dockerfile
containinginstructions to build the Docker Image, then use the Google Cloud Buildservice to build the Image and store it in the Google Container Registry.Borealis provides the
DockerTask
Firetask to run a task in Docker. Itpulls a named Docker Image, starts up a Docker Container, runs a given shellcommand in that Container, and shuts down the Container. Running in a Containeralso isolates the task's runtime environment and side effects from theFireworker and other tasks.3/8 In Percentage
Google Cloud Storage: Although you can set up an NFS shared file servicefor the workers' files, Google Cloud Storage (GCS) is the native storageservice. GCS costs literally 1/10th as much as NFS service and it scales upbetter. GCS lets you archive your files in yet lower cost tiers intended forinfrequent access. Pretty much all of Google's cloud services revolve around GCS,e.g., Pub/Sub can trigger an action on a particular upload to GCS.
But Cloud Storage is not a file system. It's an object store with a lightweight protocol to fetch/store/list whole files, called 'blobs.' It does notsupport simultaneous writers. Instead, the last 'store' of a blob wins. Blobpathnames can contain
/
characters but GCS doesn't have actual directory objects,so e.g. there's no way to atomically rename a directory.DockerTask
supports Cloud Storage by fetching the task's input files from GCSand storing its output files to GCS.Logging:
DockerTask
logs the Container's stdout and stderr, andfireworker
sets up Python logging to write to Google'sStackDriver cloud logging service so you can watch your workers runningin real time.Projects: With Google Cloud Platform, you set up a project for your teamto use. All services, VMs, data, and access controls are scoped by the project.
How to run a workflow
After doing one-time setup, the steps to run a workflow are:
- Build a Docker container Image containing your payload tasks to run inthe workflow. The
gcloud builds submit
command will upload your code and aDockerfile
, then trigger a Google Cloud Build server server to build theDocker Image and store it in the Google Container Registry.See Building Your Docker Image for how to buildthe Docker Image, starting with writing theDockerfile
. - Build your workflow and upload it to MongoDB.You can do this manually by writing a
.yaml
file and running thelpad
command line tool to upload it, or automate it by implementing a workflow builderprogram which calls FireWorks APIs to construct and upload aWorkflow
object.The workflow will run instances of theDockerTask
Firetask. Of courseit can run other Firetasks as well; they just won't get deployment,storage, logging, and timeout features fromDockerTask
.If you need to open a secure ssh tunnel to the MongoDB server running ina Google Compute Engine VM, use theborealis/setup/example_mongo_ssh.sh
shell script. - Start one or more
fireworker
processes to run the workflow tasks.There are three ways to do this:- Run the
fireworker
process locally (this is handy fordebugging) by running thefireworker
Python script. - Launch a group of fireworkers on Compute Engine VMs (this is handy forgetting a lot of work done in parallel) by running the Python script
gce
. - Automate the launching of a group of fireworkers on GCE by making yourworkflow builder code call on the Borealis
ComputeEngine
class after itbuilds and uploads the workflow.
- Run the
- While it runs and afterwards, you can:
- watch the main GCP dashboardTip: Add VM load charts to this page.
- watch the Compute Engine VM Instancesconsole to see your list of workers
- watch the Google Cloud Logs Viewerof your running workers
(The INFO level will include the big picture for running workers.The DEBUG level will also include the console output and other details for the firetasks.Filter to just your worker nodes with an advanced filter likeresource.labels.instance_id:'fireworker-USER-'
,inserting your $USER name.) - Workaround: If this command prints an error
TypeError: cannot pickle '_thread.lock' object
there's a compatibility problem with FireWorks on Python 3.As a workaround, runwhich starts the webgui server without the web browser.It prints the server URL such as http://127.0.0.1:5000.Then open a browser to that URL.(Just CMD-click the URL in iTerm.) - build a new Docker Image containing bug-fixes, then re-run Fireworks tasks.If all the fireworkers have exited, run the
gce
command to launch newfireworkers. - launch additional fireworkers:Set the BASE number argument or NAME-PREFIX argument so the new VM names won'tconflict with your already-running VMs.
- Archive or delete a workflowor reset the entire Launchpad database:
- Access the output files in GCS via thegsutil command line tool, thegcsfuse file mounting tool, and theStorage Browser in theGoogle Cloud Platform web console.Tip: Use
gsutil -m
to transfer multiple files in parallel.(If it printsTypeError: cannot pickle..
,configuregcloud
andgsutil
to use Python 3.7 per the note inInstall the development tools.)
More detail on the Borealis components
The Borealis components can be used separately. Together they enable runningFireWorks workflows in Google Cloud.
gce:The
ComputeEngine
class and its command line wrapper gce
create, tweak, and delete a group of worker VMs.After you generate a workflow, call FireWorks'
LaunchPad.add_wf()
(or run FireWorks' lpad add
command line tool) to upload it to theLaunchPad. Then call ComputeEngine.create()
(or the gce
command line)to spin up a group of worker VMs to run that workflow and pass in theparameters such as the LaunchPad db name and username(via GCE metadata fields).ComputeEngine
and gce
can also immediately delete a group of workerVMs or ask them to quit cleanly between Firetasks, although we usually letfireworkers shut down on their own after an idle timeout.ComputeEngine
and gce
can also set GCE metadata fields on a group ofworkers. This is used to implement the --quit-soon
feature.After installing the
borealis-fireworks
pip:Or to run it within the
borealis-fireworks
source code repository:fireworker:The
fireworker
Python script runs as as the top level program of each workernode.fireworker
reads the worker launch parameters and calls the FireWorks libraryto 'rapidfire' launch your FireWorks 'rockets.' It also handles server shutdown.fireworker
connects Python logging to Google Cloud'sStackDriver logging so you can watch all your worker machines in real time.To run
fireworker
on GCE VMs, you'll need to create a GCE Disk Image thatcontains Python, the borealis-fireworks pip, and such. See the instructions inhow-to-install-gce-server.txt.The
fireworker
command can also run on your local computer for easierdebugging. For that, you'll need to install the borealis-fireworks
pip and setup your computer to access the right Google Cloud Project.DockerTask:The
DockerTask
Firetask pulls a named Docker Image, starts up a DockerContainer, runs a given shell command in that Container, and stops the container.Required params:
- task
name
for logging - Docker
image
to load up into a Container - shell
command
to run in the Container internal_prefix
input/output base pathname in the Containerstorage_prefix
input/output base pathname in GCS
Optional params:
inputs
input files and directory treesoutputs
output files and directory treestimeout
for stopping the shell command, in seconds
Docker always runs a shell command in the Container. If you want to run a
Firetask
in the Container, include a little Python script to bridge the gap:Take a Firetask name and a JSON dictionary as command line arguments,instantiate the Firetask with those arguments, and call the Firetask'srun_task()
method.DockerTask
supports Google Cloud Storage (GCS) by fetching the task's inputfiles from GCS, mapping them into the Docker Container, running the task, andstoring its output files to GCS. This means you'll need to specify the inputand output paths as DockerTask
arguments.(Your workflow builder code could use this path information to compute thetask-to-task interdependencies for FireWorks.)Each path you specify in DockerTask's
inputs
and outputs
denotes a directory tree of files iff it ends with a /
.When storing task output files,
DockerTask
creates entries with names ending in/
to act as directory placeholders that speed up tree-oriented traversal.This means you can rungcsfuse without using the--implicit-dirs
flag, resulting in mounted directories that run 10x faster.DockerTask
imposes a given timeout on the task running in the Dockercontainer.DockerTask
logs the Container's stdout and stderr to a file and to Pythonlogging (which fireworker
connects to StackDriver).What's next?
See the Team Setup steps andthe Developer Setup steps.
See Building Your Docker Image.
Also see Handy Links.
Firetask 3 800
Release historyRelease notifications | RSS feed
0.7.0
0.6.6
0.6.5
0.6.4
0.6.3
0.6.2
0.6.1
0.6.0
0.5.1
0.5.0
0.4.0
0.3.3
0.3.2
0.3.1
0.3.0
0.2.1
0.2.0
0.1.1 Firefox 72 download.
0.1.0
Download files
Firetask 3 8 Trailer
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size borealis_fireworks-0.7.0-py2.py3-none-any.whl (38.7 kB) | File type Wheel | Python version py2.py3 | Upload date | Hashes |
Filename, size borealis-fireworks-0.7.0.tar.gz (246.5 kB) | File type Source | Python version None | Upload date | Hashes |
Hashes for borealis_fireworks-0.7.0-py2.py3-none-any.whl
Algorithm | Hash digest |
---|---|
SHA256 | e05d2267b06290552cbb84e3fbf778cf370c83ae031e5dd39f4e6e3cfa8ef0e0 |
MD5 | cf73a8a6f0242e1fb6f51d7fcc227232 |
BLAKE2-256 | 7d28d777dc2327a2a03f6207f1a5c674ba2ac03fd1ced2b25805cb7ffbc7ca79 |
3/8 As A Decimal
Hashes for borealis-fireworks-0.7.0.tar.gz
Firetask 3 8gb
Algorithm | Hash digest |
---|---|
SHA256 | eb2cec164f77cd136ebbcbd2d385a88d309d79effe6b5719f2f93032c9fd3de3 |
MD5 | ace42a43f5d507a70de6697dc7ce32fb |
BLAKE2-256 | 769975734506d4a80d4244208fce7d1637443a25865b88d0608519aef54f0123 |