Here's Orchestra in action based on the Getting Started guide in the documentation.
Orchestra orchestrates teams (e.g., reporters and photographers in a newsroom) and projects (e.g., a news story). In Orchestra workflows, you can assign senior experts to review other experts in order to provide feedback and iteratively refine work. Orchestra also brings automation, like classifiers or crawlers written in Python, onto projects to help out. New workflows can be added with some simple Python glue and an html interface.
Projects are a series of interconnected tasks. A project is an instance of a workflow; a task is an instance of a step.
An editor with a story about local elections would create an elections project, with tasks for a reporter/photographer/copy editor.
Tasks are carried out by an expert or by a machine.
Photographers capture the story.
Machines resize and recolor the photos.
Experts can come from anywhere, from a company's employees to freelancers on platforms like Upwork.
Core experts do the initial work on a task.
Reviewers provide feedback to other experts to make their work even better.
The core expert submits the task when their work is complete.
The reviewer can choose to accept the task, which is either selected for further review or marked as complete.
They could also choose to return the task, requesting changes from and giving feedback to the worker they are reviewing.
Certifications allow experts to work on tasks they're great at.
Experts can work toward all sorts of certifications, picking up practice tasks to build experience.
Joseph is a solid reporter but needs a little more practice as a photographer—let's give him some simple tasks so he can improve!
Experts need additional certification to work in a reviewer role.
Amy has been reporting for quite some time and would be great at mentoring new reporters.
We are a startup based in NYC that is passionate about improving how people do creative and analytical work. We have a strong team of engineers and designers who have worked extensively on systems that help people work productively online.
Beyond focusing on profit, we believe that the products and experiences we design should be considerate of their greater social context and impact. To stay true to these values, we are in the process of becoming a B-certified corporation.
Flash teams from Stanford, a study in empowering managers to coordinate interactions between a team of experts contributing to a larger project. The Foundry project is a mature research prototype of these ideas.
Active learning, or human-in-the-loop machine learning, is the study of how machine learning model training can happen in concert with a domain expert completing their work.