Chef is a popular configuration management platforms that is used to automate cluster deployments. However, Chef us designed primarily to manage individual nodes. Deploying clustered software requires additional support to apply many steps in a specific global order across all nodes in the system to ensure dependencies are respected. Karamel supports orchestration using both cluster definition files and a Chef cookbook-specific Karamelfile to instantiate a cluster and orchestrate its deployment.
In Karamel, we parallelize as much as possible of the installation of software on nodes, so that clusters come up quickly. This is achieved by dividing deployment into three steps:
- create vms (optional),
- install recipe
- other recipes
Virtualized deployments first create VMs using JClouds (for AWS, GCE, OpenStack). Bare-metal installations skip this step. Then Karamel runs the install recipe for all software artifacts (Chef cookbooks). Install recipes are written by developers such that they can execute in parallel on all machines. Finally, each Chef cookbook should contain a Karamelfile definining its orchestration rules. These rules are used to build a number of Directed Acyclic Graphs (DAGs) defining the dependencies between other recipes. Different DAGs can be executed in parallel, but execution order for recipes is managed by Karamel within a DAG. Karamel transparently handles failures due to network problems or recoverable software problems on nodes.
How does Karamel compare with Cask Coopr?
The differences between Caskdata and Karamel are as follows:
- Karamel is a lightweight standalone app that works on windows/linux/mac. CaskData is a 3-tier web server app for linux that requires a Database server.
- Karamel supports Karamelfiles - orchestration rules - in every Chef cookbook - orchestration rules are composable.
- Karamel has GitHub integration - cluster definition files are small, and you can update parameters from the UI, as parameters are downloaded from GitHub.
- Karamel only supports AWS, GCE, OpenStack, and Bare Metal. CaskData also supports Azure, Joyent, and DigitalOcean.
- Karamel has ready made cluster definitions for Spark, Flink, and Apache Zeppelin.