Using War Overlays with Crafter Engine

Crafter Engine, the delivery component of Crafter CMS is completely programmable with scripted Groovy.  You never have to write a lick of Java if you don’t want to.  Even dependencies can be managed with Ivy and Grapes.

That said, from time to time:

  •  you may find that you want to build classes in Java directly
  • or you may need to include dependencies in a deployment where Ivy and Grapes are not an option
  • or you may need to modify the Web.xml of the engine WAR file for some reason.

These scenarios are an exception but they do come up. For these scenarios, you want to create a Maven WAR Overlay.  Overlays allow you to add and override contents of Crafter CMS component WARs like Crafter Studio, Engine, Profile, and Social.

An overlay is a very simple maven project that downloads a specific version of Crafter Engine (specified in the POM file), downloads the additional dependencies you require, builds your source code that’s specific to your project, packages it to a jar and then combines all of these into a new WAR file.


Let’s create an example where we simply want to overlay a dependency into the jar, for example, the Amazon AWS SDK

Step 1: Create a project structure

Create a directory structure as follows

+--my-project (project root directory)
            +--java (your class structure starts here)
            +--webapp (any files you want to override or include in the webapp like web.xml)

Step 2: Create your Maven POM file

In a file at my-project/pom.xml put the following contents:

<project xmlns="" xmlns:xsi="" xsi:schemaLocation="">






Note that the above POM file is very simple.  It simply states that you want to download Crafter Engine 3.0.8, Download Amazon’s 1.11.x SDK and then recombine these into a new Engine WAR file called ROOT.war in the output directory target folder.

Step 3: Run the Build

Type the following command in your project directory: mvn clean package

Similar output to the following is expected:

mvn clean package
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building craftercms-engine-overlay 2.2.8-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ craftercms-engine-overlay ---
[INFO] Deleting /Users/rdanner/code/test-war-overlay/target
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ craftercms-engine-overlay ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/rdanner/code/test-war-overlay/src/main/resources
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ craftercms-engine-overlay ---
[INFO] No sources to compile
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ craftercms-engine-overlay ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/rdanner/code/test-war-overlay/src/test/resources
[INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ craftercms-engine-overlay ---
[INFO] No sources to compile
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ craftercms-engine-overlay ---
[INFO] No tests to run.
[INFO] --- maven-war-plugin:2.1.1:war (default-war) @ craftercms-engine-overlay ---
[INFO] Packaging webapp
[INFO] Assembling webapp [craftercms-engine-overlay] in [/Users/rdanner/code/test-war-overlay/target/ROOT]
[INFO] Processing war project
[INFO] Processing overlay [ id org.craftercms:crafter-engine]
[INFO] Webapp assembled in [780 msecs]
[INFO] Building war: /Users/rdanner/code/test-war-overlay/target/ROOT.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.658 s
[INFO] Finished at: 2018-03-07T21:11:09-05:00
[INFO] Final Memory: 14M/309M
[INFO] ------------------------------------------------------------------------

 Step 4: Deploy Your New WAR

In the project folder, you will now see a target folder with a ROOT.war in it.  This is your new WAR file.  You can now place this in the webapps folder of your Crafter CMS authoring or delivery server.


Content Management Meets DevOps (Part 2 of 2) How a Git-based CMS Supports Continuous Integration and Delivery

As we learned in Part 1 of this series: Content Authoring and Publishing; development and content authoring are both a major part of producing today’s digital experiences. Unfortunately, development support is not something traditional CMS platforms handle very well at scale. Crafter CMS, a 100% open source project not only supports authors but DevOps as well.

In this article, we’re going to expand on the authoring workflow and publishing mechanics we looked at in the last article and explain how Crafter CMS seamlessly supports DevOps tools and process.

Development Requires Environments

The CMS where authors work is typically considered a production system. That’s not to say that the edits they make are immediately live. Typically CMS platforms support a concept of a “draft” type workflow with the ability to publish approved content and make it “Live”.

Crafter CMS is no exception here. As we described in our last article in this series, Authors work in a decoupled authoring environment and content is published from authoring to delivery on approval.

In the authoring environment, content can be edited and previewed via our web-based authoring tool called Crafter Studio. Edits are saved in a Git-based repository called “Sandbox”. On approval, edits are moved to a repository called “Published.” Crafter Studio handles all of the mechanics of making editing and publishing safe and simple for authors. Authors simply click edit and make changes, perform reviews and publish work via the UI. In the background, Crafter Studio will lock content, perform commits and take any other actions necessary to perform the low-level repository tasks necessary.

The authoring environment is akin to a work and test area for the authors. Relative to the live site or app the content authors are deploying content to, the Crafter Studio (and the Sandbox repository) where they edit, preview and approve content is what we would call a lower environment. Work is promoted from the lower environment (authoring) to the live environment (delivery.)
Developers almost always work in lower environments. No real developer modifies the code in production. Experience shows that making changes in production leads to outages and hinders future upgrades. Interestingly, many headless CMS platforms making their way to the market today require developers managing content types to work directly in production! While the scope of development in these systems is limited, the end results are the same.

In Crafter CMS, because we decouple authoring from delivery, it is possible for developers to work directly with the production system via authoring. Changes can be made and tested in the authoring environment before being pushed live.

Figure 1: Production environment.  Showing DevOps as part of the production workflow.  DevOps do not want to touch production directly and should not have to.  Unfortunately, most CMS platforms force them to do so.

That said, developers do not want to work with CMS tools! They want to work with their own Integrated Development environments (IDE) and other tools. They want to work locally on their own machines where they have control and can debug. They want the ability to collaborate with their teammates on feature branches. They want to be able to work in parallel with other development teams without fear of stepping on each other or interfering with content authoring activities and they want to be able to easily move their work through the various environments of their certification and testing process out to production. Developer environments and process are complex.

To illustrated this I have included a diagram below of a typical developer process and set of environments which must be followed in order to get new functionality to production.

Figure 2: Typical DevOps workflow.  Today’s modern app development DevOps workflows include parallel development efforts on the same project and extend from the developer’s local environment and team sandbox all the way to production.

Traditional CMS Systems and Environments, Code Artifacts and Process Don’t Mix Well

It is difficult at best to support this kind of process with traditional CMS architecture. Server-side code is typically managed in a source code repository while CMS related code artifacts like templates, Javascript and CSS (that should be managed in a source code management) want to live in the CMS. Server-side code typically requires heavy packing and deployment systems like JARs, WARs and OSGi bundles. It takes a lot of effort on the part of the developer to get an environment setup locally so that they can develop and it’s equally difficult to get a specific set of code into one of the often many environments. Worse, content is always stored in a JCR repository or SQL database and it’s a huge burden to get the latest production content to a lower testing environment. The whole process of “code forward, content back” with traditional CMS is a nightmare for DevOps. Still worse, in some environments, going to production means interruptions in content authoring ability for content authors while their environment is updated with the latest code. These outages can last hours to days or in some cases even longer.

These conditions are unacceptable.  This is no way to support innovation at a time when the amount and speed of innovation an organization can sustain is a major key to competitive advantage.

Crafter CMS Provides the Solution

Crafter’s Git-based dynamic CMS tackles all of these challenges head-on with a set of technologies that incorporate lightweight development, integration with developer tools and process and elastic scalability that provides the ability to serve any front-end technology via API or markup with fully dynamic content.

First: Crafter CMS supports a fully featured, Spring-enabled, Groovy-based scripting layer. This is a subject for a whole blog of its own — but without getting too in-depth, it’s huge. You can create classes on your own in Groovy. You can leverage classes in the classpath. You can create your own beans based on Groovy or Java classes, you can inject existing beans and services. It’s a rich, robust powerful environment full of services for you to build on at the speed of scripting in the same languages and frameworks that you are used to.
Second: Crafter CMS sits on top of Git. Git is traditionally speaking, a source code version control system. That means that all of your code can live in the same version control system as the business content, together. It’s versioned together and it can be packaged and deployed together. Crafter CMS makes the world of CMS simple for content authors and developer native for DevOps.
Third: Because Crafter is Git-based you can natively support your ideal DevOps process for Continuous Integration and Delivery (CI/CD).


Let’s jump in and take a look at how it works.
Below is what you might think of as the archetype of a developer workflow that’s integrated with the production authoring environment. In this diagram, you will find a mechanism for both a “code forward” activity in which code can be moved through a development process through to production as well as a “content back” activity in which production content, including in-process content, can be brought back to lower development environments! including local development machines.

Figure 3: Archetypal DevOps workflow that describes in general how content flows back to the development process and code flows forward through the certification process to production.

The above diagram has a lot going on. Let’s break it down and explain things a step at a time.

Step 1: Create an Upstream Repository

In order to facilitate a developer workflow, you want to establish an upstream repository for your Sandbox in your production authoring environment. Any Git repository will work. It’s typical to use a Git repository that has a UI/Workflow atop of it such as Github, Bitbucket, Gitlab and other similar systems. The upstream repository is the root of the developer workflow.

Figure 4: A Git repository with support for “developer/team workflow” such as Github, GitLab or Bitbucket is configured as a remote upstream repository of the Production Authoring Sandbox repository.  This is the primary point of contact between the authoring process and the development and code release process come together.  

Step 2: Use Automation to Sync Sandbox with the Upstream

Note that the repository is what the Git community refers to as an “upstream” remote repository. That means in a sense the Production Sandbox becomes subordinate to it from a Git log perspective. This upstream repository is where content and development work will be merged before making its way to the production Sandbox and ultimately to the Published repository and the delivery nodes themselves. Also, note that nothing changes about the publishing and deployment configuration and topology of the Production environment.

You will want to keep the upstream repository up to date with authoring. The best way to accomplish this is to use a DevOps automation platform such as Jenkins or Bamboo to orchestrate a push to the upstream on a schedule. Keeping the upstream repository up to date with the authoring Sandbox repository provides downstream development and testing repositories with the latest content and helps to avoid conflicts when it’s time to promote from the upstream repository to the authoring Sandbox repository.

Figure 5: Content flows from Production Authoring Sandbox to the Developer repository via automated Git Push

Step 3: Use the Upstream Repository to Support Development and Testing Environments and Workflow

From this upstream repository, you will feed all lower environments and branch for each of your development efforts. From the upstream repository, you can support any development process and supply content to any development or testing environment. Lower environments may consist of Authoring and Delivery, or just Authoring, or just Delivery. It depends on the needs. For example, Development often contains both Authoring and Delivery, while QA tends to focus only on Delivery. Simple “Git pull” mechanics are used to move code and content from the upstream repository, typically from a branch.

Figure 6: Development branches and forks can be created to meet your specific workflow needs.  Lower environments use these repositories and branches as their upstream.

Step 4: Merge Code Updates Into the Upstream Master and Pull them to the Production Authoring Sandbox Repository

When you are ready to move code from a development branch to the authoring Sandbox you will first merge the work into the master of the upstream repository. You can do merge all of your commits or you can merge them into a single commit by using the Git rebase command. By Squashing all of the commits into a single commit you make it easier to move the workaround as a single unit. Merges are typically done via pull requests on repositories that support development workflow.
Once the merge operation is complete in the upstream repository your automation will carry that work to the production authoring Sandbox.

Figure 7:  Use “Pull Requests” and other Git workflow mechanics to promote code through the process.  When it’s ready to go live merge it to the “Master” of the Developer Repository.  At this point, it will flow via automation to the Production Authoring Sandbox.

Step 5: Use Crafter Studio’s Publish Commits API to Publish Code to Delivery

Once your development work is in the Production Sandbox you will want to publish it. To do this Crafter Studio provides a REST API that will trigger the publishing on one or more commits. Simply call Crafter Studio’s Publish Commits REST API (/api/1/services/api/1/publish/commits.json) via the DevOps automation platform passing the required parameters including the commit IDs to be published and Crafter Studio will move the work to the Published repository from which it will be replicated to your delivery nodes.

Figure 8: Move code from Sandbox to Published with a simple API call. 

Step 5: Build Tons of Amazing Things

Now that you have the basic mechanics of Crafter’s native Git-based distributed repository and development workflow you are ready to support any amount of parallel development you want with the kind of tools and process rigor you are used to. Because Crafter’s Git-based repository is distributed, authors and developers, even developers working locally are all working out of what is essentially the same repository.

The workflow enables the team to collaborate and work simultaneously without ever interrupting or stepping on each other’s toes. It’s time to start innovating!

How do I set up this workflow?

For the purpose of simplicity, we’ll assume you’ve got Crafter Studio up and running and you are about to create a new project.

Step 1: Create your upstream

Create an empty repository in your upstream (GitHub, Bitbucket etc.) The specific steps depend on the repository you are using. The key here is that you want to create an empty or what’s sometimes called a “bare” repository.

Figure 9: Create a bare repository in your developer Git. In this example, we’re using Github.

On Github, once created you will see the following screen. Here you will note the repository URL ( which you will need for the next step. Also, if you’re trying to create an upstream for an existing project (out of scope for this blog), you’ll find the instructions below in the “push an existing repository from the command line” section.

Figure 10: Bare repository created.  

Step 2: Create Your Project In Crafter Studio

Next, you want to log in to Crafter Studio as the admin user. The admin user has the rights to create new projects (called sites.) Click Create Site.

Figure 11: Create site via Crafter Studio

Clicking Create Site will present you with the Create Site dialog. This dialog changes depending on what you choose. Below is an example of the dialog filled out in a way that creates your project locally, set the Github repository as its upstream remote and pushes the initial project contents to the upstream repository.

Let’s walk through each part of the dialog:

Figure 12: Create Site Dialog in Crafter Studio, populating a bare upstream Git repository.

  1. The first thing you need to do is give your site an ID. The ID itself doesn’t matter in a sense. It doesn’t need to match anything per se, technically speaking the only requirement is that it’s unique. That said, it’s a best practice to provide an ID that is meaningful/recognizable to the team. If your website is called a good ID might be “freshflowerscom”
  2. Next, because you plan to connect this project to an upstream repository you want to click the plus (+) on “Link to upstream remote Git repository” This will open a number of new fields.
  3. In the “Remote Git Repository Name” field you want to provide a repository name that makes sense. It’s common to use “origin” or “upstream.”
  4. In the “Remote Git Repository URL” field you must provide the link to the Git repository discussed in Step #1:
  5. Provide your credentials in Git Remote Repository Username and Password
  6. Choose the option: “Create site based on blueprint push to a remote bare repository.” This means that Crafter Studio will create a new site based on the blueprint you choose, link the remote repository as an upstream and then once the blueprint is installed in the local Repositories it will be pushed automatically to the upstream remote.
  7. Choose your blueprint. There are several out of the box blueprints provided by default. Choose one of these or one of your own. For our example, we’ll choose Empty which is the “Hello World” of blueprints.
  8. Click Create. Crafter CMS will create the local repositories, Solr core and internal data structures required to support the project and install the blueprint. Once complete it will connect to the upstream and push the contents of the Sandbox repository to the remote.

Figure 13: Site is created and the contents of the sandbox are automatically pushed to the upstream repository.

Step 3: Set up Your Delivery Nodes

Now that your project is created you can set up the rest of your production environment by initializing your delivery nodes to receive deployments from authoring. Remember these delivery nodes will pull from Crafter Studio’s repositories, not the upstream remote repository.

When you add a new delivery node a simple command line script is run on that node that configures it to replicate and process content from the “Published” repository from authoring.

  • Instructions for creating a site can be found here.
  • Instructions for initializing a delivery node can be found here.

Step 4: Set up your Developer Workflow and Lower Environments

Now that your upstream repository is initialized in GitHub you can set up any developer workflow you want. It’s typical to consider Master to be in-sync with the Production Authoring Sandbox. Given that, you don’t want to work in Master. Create branches to isolate development work from work that’s ready to move to Production Authoring. Below is an example topology that shows multiple environments and developer workflow that include feature branches, developer forms, and local developer clones.

Figure 14: Full DevOps “Code Forward, Content Back” workflow for CI/CD with Crafter CMS leveraging Git mechanics and DevOps automation.

I Am a Developer, I Want to Work Locally Against The Upstream

Let’s consider for a moment that your a new developer joining the team. The topology above is already set up and you just want to get a local environment up and going. Simple. Follow these instructions.

  1. Install Crafter Studio locally (Source build or Binaries bundle)
  2. Login as Admin
  3. Click Create Site
  4. Fill out the Create Site Form as in a similar fashion described in Step 2, except this time you chose the option to create your site based on an existing upstream repository. This can be your team’s branch or your own fork. The exact workflow is up to you.

Figure 15: Set up a project based on existing remote Git repository via Crafter Studio


Today’s digital marketplace is constantly evolving. Companies are always iterating on existing functionality with improvements and deeper integration or introducing new functionality and channels for their employees, partners, and customers. Crafter CMS provides the right technology and integrates seamlessly with your DevOps processes to enable you to achieve a high, sustainable continuous rate of constant, iterative development, integration and delivery (CI/CD.)

Digital experience teams finally have a toolset that allows authors to work continuously without being interrupted by developers.

Developers have a means for easily moving code forward through environments and pulling content back from production to lower environments.

Further, with Crafter’s distributed repository development shops can run as much parallel development as they want and developers are able to leverage the workflow and local tools they are accustomed to.

“Code Forward, Content Back” is a trademark of Crafter Software Corporation

Content Management Meets DevOps (Part 1 of 2) How a Git-based CMS Improves Content Authoring and Publishing

Traditional CMS platforms based on SQL and JCR repositories have begun to show major signs of weakness in keeping up with today’s demands for a high rate of innovation and rapid scalable deployment on modern elastic architectures. This is nowhere more evident than the move towards headless CMS. Many CMS platforms today push headless, or what some call Content as a Service (CaaS), as the one-stop-shop solution to the struggles most CMS platforms have in providing support for scalability, multi-channel, and development integration. It’s not. Headless capability is important but it has its own limitations.

Crafter CMS, an open source Git-based dynamic CMS tackles all of these challenges head-on with a set of technologies and that incorporate lightweight development, integration with developer tools and process, and elastic scalability for content delivery that provides the ability to serve any front-end technology via API or markup with fully dynamic content.

In this two-part series, we’ll explain the basic mechanics that support content authoring, publishing and developer workflow and demonstrate how these mechanics combined with Crafter’s architecture and developer stack set a new standard for what a CMS can provide in today’s competitive digital marketplace.

Content Management and Deployment Mechanics

In this section we’ll explore the mechanics of how (non-technical) content authors work with the CMS and how their changes, once reviewed and approved, are deployed from their authoring tools to a live content delivery system.

Crafter CMS is decoupled, composed of several microservices where content authoring and content delivery services are separated into their own distinct, subsystems. This model has many advantages related to security, scalability and delivery flexibility. In a decoupled architecture, content is published from authoring to delivery as shown in the diagram below. The delivery system may be any number of independent digital channels – enterprise website, mobile app, social, augmented reality, digital kiosk or signage, e-commerce front end, microsite, etc.

Crafter CMS supports authoring via Crafter Studio that sits on top of a headless Git-based repository and publishing system. Content authors don’t need to know anything about Git. They simply work with Crafter Studio, a web-based application. Crafter Studio provides beautiful content entry forms, in-context editing with multi-channel preview, drag-and-drop layout, component placement, image cropping, and more. While content authors are performing their work, Crafter is managing all of the Git mechanics, managing locking, creating a time-machine like, Git-based version history and audit trail for them behind the scenes, all accessible to them via the Studio UI.


Figure 1: Crafter CMS microservices applied to decoupled architecture

Crafter’s publish mechanism deploys content from the Authoring system to the Delivery system. Content logically flows from the authoring environment to the delivery environment. The mechanism for this, given the underlying Git repo, is a “pull” type interaction. Meaning the actual network conversation is initiated from the delivery infrastructure to the authoring infrastructure, as shown in Figure 2.

Each delivery node has a Deployer agent that coordinates deployment activities on the node for each site that is being delivered on that node.

  • Delivery nodes can initiate deployment pulls either on a scheduled interval (a “duty cycle”), on-demand via an API call, or both.
  • The Deployer performs a number of activities beyond receiving and updating content on the delivery node. A list of post-commit processors is run. These can be used to execute updates on search indexes, clear caches and perform other such operations.
  • The Delivery node maintains a clone of the Authoring Git-based repository.
  • The Crafter Deployer takes care of managing the synchronization of the delivery node’s clone authoring repository from the authoring environment.
  • Git-mechanics ensure content sync is 100% accurate.

Figure 2: Crafter’s Dynamic CMS Publishing via Git

Technically speaking, Authoring does not require knowledge of the Delivery nodes. This makes the architecture more elastic, globally scalable and even enables Crafter to support disconnected and intermittent content delivery.

  • Elastically add new nodes or revive dormant nodes and they will sync to the latest without any additional wiring.
  • Create region-based depots to avoid transferring data more than once over long distances for global deployments.
  • Airplanes, cruise ships, drilling/mining locations and other remote disconnected deployments can operate with their latest pull of content, and sync up with Authoring when connectivity is available.

Figure 3: Elastic Delivery

In Crafter CMS, only approved content is published to the delivery environment. Crafter manages this by using 2 repositories for each project. One called a “Sandbox” which contains work-in-progress and the other called “Published” which represents approved, published work and complete content history.

  • Authors use the Crafter Studio UI to review and approve content via workflow.
  • Crafter Studio takes care of moving approved work between Sandbox and Published repositories.
  • Delivery nodes monitor the published repository for updates.

Figure 4: Authors work in Sandbox. Delivery nodes pull from Published.


Crafter’s Git-based publishing model provides your authoring team with a highly reliable, highly accurate publishing mechanism that is elastically scalable, globally distributable and supports multi-channel.  Crafter CMS’s architecture enables your team to reliably deliver your dynamic content on any channel, wherever and whenever it is needed.

Further, As we’ll see in in Part 2, this architecture enables content authors to work side-by-side with DevOps, while they continually introduce new features and functionality without any disruption to the authors.

How do I set up this workflow?

The underlying Git repositories and related workflow for Authoring require no setup at all. When you create a project in Crafter Studio it automatically creates the local “Sandbox” and “Published” repositories. When you add a new “Delivery” node a simple command line script is run on that node that configures the node’s deployer to replicate and process content from the “Published” repository from authoring.

  • Instructions for creating a site via Crafter Studio can be found here.
  • Instructions for initializing a delivery node can be found here.


Content authors are non-technical users who need powerful but easy-to-use tools to help create, maintain and manage their digital experiences. Crafter Studio provides these users with a web-based application that makes it easy for content authors to achieve their goals. Under the hood, Crafter Studio leverages a powerful Git-based repository and deployment engine that provides authors with next-generation versioning and auditing mechanics as well as robust, elastic and distributed deployment.

Today’s digital marketplace is constantly evolving. Companies are always iterating on existing functionality with improvements and deeper integration or introducing new functionality and channels for their audiences. For today’s most innovative and competitive organizations, ongoing development and the move to DevOps is a fact of life. The companies that have the greatest success are those that have the right technology and processes through which they are able to achieve a high, sustainable continuous rate of constant, iterative development, integration and delivery, i.e., Continuous Integration and Delivery (CI/CD).

In the second half of this blog series, we will take a deep dive into how Crafter CMS seamlessly integrates with your CI/CD processes to enable your entire team of developers and content authors to innovate collaboratively without interfering with each other’s workstream.


How To Change the Ports on Your Crafter CMS Installation

One of the most common questions I get from developers is:  “When I download Crafter CMS it runs on port 8080.  How do I change that?”  They are not simply looking to put Crafter on port 80 or 443.  They want to move it to port 9080 or some other port because they are already running something on port 8080.

To make things easy, let’s look at a standard development installation — which consists of the following microservices:  Crafter Studio, Crafter Engine, Crafter Search, Solr and Crafter Deployer.

In the diagrams above you will note the black arrows between components.  These are HTTP connection to (typically) localhost and the port specified on the target component.    Let’s review each of these connections:

A. Developer/consumer goes to Crafter Studio application (/studio).  Crafter Studio IFrames Crafter Engine rendering.
B. Crafter Studio queries Crafter Search when users do a search inside the CMS
C. When rendering Crafter Engine can leverage Crafter Search to perform content queries and searches.
D. Crafter Search applies platform-specific business rules and makes query requests to Solr via connection D
E. When content, code or configuration is saved via Crafter Studio or directly via Git it is picked up by the preview deployer and published to Crafter Search.  Crafter search performs inserts updates and deletes on Solr via connection D
Crafter Studio maintains/caches project/user and operational metadata (workflow state, dependencies) about content locally in an embedded MariaDB.

Configuration for Tomcat ports:

  • Impact: Ports for Crafter Studio, Crafter Engine, and Crafter Search
  • Location: INSTALL_DIR/bin/apache-tomcat/conf/server.xml
  • Configured ports: There are several ports listed in this XML file.
    • 8005 (shutdown port),
    • 8080 (HTTP connector)
    • 8443 (HTTPS connector)
    • 8009 (AJP connector)
  • INSTALL_DIR/bin/ / crafter.bat
    • Linux: export TOMCAT_HTTP_PORT=8080
    • Windows: SET TOMCAT_HTTP_PORT=8080

Once you change the Tomcat ports you must update the configuration for the communication between microservices to Crafter Search:

  • Impact: Crafter Studio Connections
  • Location: INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/studio/extension/studio-config-override.yaml
  • Configured Ports:  There are several ports to update when Tomcat’s connector ports are changed:
    • studio.preview.engineUrl
  • Impact: Crafter Engine
  • Location: INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/engine/extension/
  • Default Ports:  Update the crafter search port
  • Impact: Crafter Deployer
  • Location: INSTALL_DIR/bin/deployer/config/classes/crafter/engine/extension/base-target.yaml
  • Default Ports:  Update the crafter search port
    • target/search/serverUrl

Configuration for Deployer ports:

  • Impact: Configures the ports for the deployer, impacts Studio
  • Location: INSTALL_DIR/bin/deployer/config/application.yaml
  • Configured Portsport
  • INSTALL_DIR/bin/ / crafter.bat
    • Linux: export SET DEPLOYER_PORT=9191
    • Windows: export DEPLOYER_PORT=9191

Once you change the Deployers ports you must update the configuration for the communication between Crafter Studio and the deployer

  • Impact: Crafter Studio’s communication with deployer
  • Location: INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/studio/extension/studio-config-override.yaml
  • Configured Ports:  There are several ports to update when the deployer ports are changed:
    • studio.preview.defaultPreviewDeployerUrl
    • studio.preview.createTargetUrl
    • studio.preview.deleteTargetUrl

Configuration for Solr ports:

  • Impact: Crafter Search’s communication with Solr
  • Location: INSTALL_DIR/bin/ / crafter.bat
  • Configured Ports:
    • Linux: export SOLR_PORT=8694
    • Windows: SET SOLR_PORT=8694
  • Impact: Crafter Search’s communication with Solr
  • Location:INSTALL_DIR/bin/apache-tomcat/shared/classes/crafter/studio/extension/
  • Configured Ports:

Five Reasons Why You Should Use a Git-based CMS (Part 3 of 5)

Most CMS technologies are what we would call a “coupled CMS.”  The content authoring and content delivery environments are usually part of the same stack. The act of “going live” with new content or a feature is essentially based on the act of marking a true/false in a database field. There are a lot of problems with coupled CMS platforms around security, performance, scalability, and flexibility (you can learn more here.)

For these reasons and many others, Crafter CMS is built as a decoupled CMS. With a decoupled CMS you author content in one system and publish to another separate system. For platforms like Crafter CMS that are decoupled, when correctly implemented, the architecture provides great solutions for the issues mentioned above. That said, nothing is without its challenges. Decoupled systems, by their nature, are typically very scalable and can have many instances all over the world. Security, scalability, and distribution are no longer issues that only concern the Internet’s biggest players like Google and Amazon.  Security and distribution impact customer experience, safety and help reduce operating costs.  Every brand-conscious and customer-forward organization in the world is focused on these tactical issues.

Once you have a decoupled, distributable deployment model, the challenge becomes making certain that the content on the servers all over the world is the same — everywhere. Every decoupled solution has an approach for this. Some better than others. That said, few if any of the approaches offered out of the box by today’s traditional CMS platforms “mathematically” ensure every remote instance is 100% up to date and in sync with every other instance. If there are bugs in the deployment code or there is trouble in the environment you may get out of sync.

In our previous posts, we looked at Crafter CMS and its Git-based versioning (part 1) and distributed repository (part 2).  In this post, we’ll take a deeper dive into how Crafter CMS leverages Git mechanics to provide a better, more consistent distributed publishing mechanism.

Reason #3: Distributed, scalable, consistent publishing

Crafter CMS uses Git mechanics to publish content to its decoupled delivery space. When Git reports that its repository is set at a specific version that means that every file is guaranteed to be present and in the proper state for that version, it is. Fact. It’s provable.

The reason it’s provable is due to the fact that the Git mechanics that underlie Crafter CMS’ content repository are based on Git’s purely functional data structures. “The main difference between an arbitrary data structure and a purely functional one is that the latter is (strongly) immutable” (Wikipedia).  What this means is that as commits happen within the repository an entirely new immutable data structure is created containing the changes for the commit. No action is taking on the previous data structure(s.)  Nothing you ever change can be lost or corrupted by an operation once the change has been committed.  Moreover, in Git, the ID for the commit is essentially a SHA1-hash of metadata and the content in the directory tree. By definition, if a single bit changes anywhere in the tree a new SHA1-hash must be generated.


While this explanation is an oversimplification of Git’s algorithm, it is essentially the model of how it works. The point is that two repositories on two different machines with the same commit ID are mathematically guaranteed to be the same. That’s an extremely useful mechanism for versioning but it also is a very large helping hand in publishing. Crafter CMS publishes (replicates) content based on Git commits. If you want to know if an endpoint on the other side of the world is the same as what you expect, you only have to compare the commit ID(s).

In today’s elastically scalable, globally distributed world you can have any number of servers.  You need a means to make sure they are all in sync. As you can see above, Git’s internal mechanics give us just that.  Crafter CMS is the first decoupled CMS with the capacity to scale geographically across an elastic cloud and at the same time make 100% certain that remote instances are consistently running the same version of content and code.


Decoupled CMS platforms provide push-based publishing, offer greater architectural flexibility and are much easier to scale elastically and distribute globally.  Along with this increased power and flexibility comes a need to ensure that all remote endpoints are in sync with one another and are up to date.  While this problem is solvable, few of today’s decoupled CMS platforms provide a solution for this that is 100% guaranteed and mathematically verifiable.  Crafter CMS and its Git-based repository leverage Git mechanics for publishing and replication to remote nodes.  Calculating changesets and verifying that an endpoint is in a particular state is based on the proven algorithms and data structures that back Git, the world’s most powerful and popular distributed source repository.
Stay tuned for our next blog entry to learn another major reason why you should use a Git-based CMS!

Work Your Own Way with Crafter CMS (Series Part 1): Step-through Debugging

Most CMS platforms do a decent job of simplifying content and digital experience creation and editing for non-technical content managers.  The challenges really start once you need to innovate and development is required.  Traditionally CMS platforms have been pretty bad for developers.  They require a lot of CMS specific knowledge and don’t integrate with developer tools and process.
Here are 7 things that developers really want with a CMS:

  1. Let me work locally with my own tools like my IDE and my source code management
  2. Let me leverage my existing skills.  I want a low learning curve. Don’t make me learn a new, niche framework
  3. Let me work in teams on multiple projects at the same time without interfering
  4. Let me maintain a real development process
  5. Make the integration with the CMS seamless
  6. Don’t make me do builds
  7. Don’t make me do heavy deployments

In this installment of the Work Your Way Series we’re going to tackle item #1 (Let me work locally with my own tools like my IDE and my source code management.)   Let’s start with some background: Crafter CMS uses Git as its primary content store.  That’s the foundation of the solution for developer desire #1. A developer can mount a local clone of a Crafter CMS project directly with their IntelliJ, Netbeans, Eclipse or other IDE.  That means they can use their preferred development tools to edit and debug code and templates.  And as they work, all of the changes they make are tracked by the Crafter CMS via its native Git support.  Sounds awesome right?  Let’s learn how to get set up:

Step 1: Get a local copy of Crafter CMS running

You are going to use your IDE to update and debug your code and templates.  You’ll want a local instance of Crafter CMS running so you to execute the code and attach your debugger to.

To install and get Crafter CMS running (authoring environment) locally follow this guide:

Step 2: Start a Crafter CMS project

Now that Crafter CMS is up and running let’s create a project so we have a place to work.  We’ll create a local project for the sake of this article, however, as we’ll learn later in this series, it’s possible to clone a remotely managed project as well.

To create your CMS project follow this guide:

Step 3: Mount your IDE on top of your Crafter CMS project

For this article, I’m going to use IntelliJ.  Any IDE that allows you to connect to a remote JVM via Java Debug Wire Protocol (JDWP) should work.  Eclipse and Netbeans are other solid IDE options.

  1. Within IntelliJ, create a new project:

  • Select Create New Project


  • Choose Groovy

  • A: Configure your project name to something you like
  • B: Browse to the Git repo “Sandbox” under your “CRAFTER-INSTALL/data/repos/sites/PROJECTID/sandbox”
  • C: Change the module name from Sandbox to your project name


  • Once your project is created, open it to see all the files.  IntelliJ automatically recognizes you are sitting on top of a Git repo and will allow you to do Git operations right from the IDE.
  • Note that a Groovy project creates an “src” folder.  You can delete this.  You don’t need it.

  • Right click on your model and add a new file called “.gitignore”
  • Do not add this file to Git when prompted.

The contents of the .gitingore file should be as follows:

  • This will tell Git and Crafter CMS to ignore your IDE project files

  • If you were to run a “git status” command in your project sandbox you would note that Git is totally satisfied and ready to rock 🙂


Step 4: Set up step-through debugging

Great, now that we’ve got our IDE sitting on top of our project and we’ve got the Git integration configured we’re ready to set up the step through-debugging.

  • Shut down Crafter CMS
  • Add the following lines to CRAFTER-INSTALL/bin/apache-tomcat/bin/ (or setenv.bat)
    • Syntax in .bat will be slighly different

export CATALINA_OPTS="$CATALINA_OPTS -agentlib:jdwp=transport=dt_socket,address=localhost:8000,server=y,suspend=n"
  • Start Crafter CMS

Back in our IDE we can now configure and start our remote debugging session!

  • Go to “Run” and select “Debug”

  • Since this is a new project we’ve got to configure our debugging connection.  The next time you click Debug you’ll just select the existing configuration and click Run.

  • Click the + (plus) icon to create a new configuration
  • Select “Remote”

  • Give the debug configuration a name
  • Change the port from 5005 to 8000 (to match the value in
  • Click “Apply” to accept the changes
  • Click “Run” to attach to the JVM running Crafter CMS

  • Congrats! If your JVM configuration is correct and your ports match in the debug config when you click “Run” you’ll see the “Connected” message in the console as shown above.

Step 5: Start debugging

Now that we’re connected we can start stepping through code in our Groovy Scripts.

  • Open the “Scripts” folder in your project
  • Browse down through classes, org, craftercms, sites, editorial and open SearchHelper.groovy
  • Set a breakpoint at line 56.  This method is used by the Home Page controller of the editorial website we created in step 2.
  • In a browser, open the Crafter CMS preview for the homepage of the website we created

  • When you load the homepage in Crafter Studio the rendering will pause

  • IntelliJ’s window will typically come to the foreground automatically.
  • The thread has been paused and the IDE is now in control of the thread execution.
  • You can now use the step-through debugging tools in the IDE to walk through the code.

Step 6: Magic

Found a bug?  Here’s where things get really fun.  We’re about to see the benefit of bringing Git, Groovy and your IDE together in one place.  Fixing the bug and sharing that fix with the rest of your team is a breeze:

  1. Once you understand the bug and you know what code you want to change, click Play to let the thread complete.
  2. Using the IDE, fix the Groovy code.
  3. Simply refresh the browser again to test.  Step through and verify things are working.
  4. Using the IDE to interface with Git, commit the code and push the code forward in your team mates.
  5. Find something fun to do or work on.  You have a lot more time on your hands now that you are working in a truly integrated, no-compile required, easy to code and debug CMS 🙂



Why Developers Should Care About CMS

As developers, we’ve got a strong handle on how to manage and deploy our code assets.  Yet every one of us, at some point in our application build has said, “What about this text? What about these images? Where do these belong?”  That’s pretty universal.  Nearly every single application today has content in it.  Be it a web app or a native app; it’s full of strings, images, icons, media and other classes of content.

This content doesn’t really belong in our code base — because it’s not code.  These non-code assets make us as developers pretty uneasy.  We know that at some point a business user is going to ask us to make a change to one of those strings and we’re going to spend hours of build and deploy cycles to handle a 30-second code change.  We know that at some point we’re going to need to translate that content.  We know at some point we’re going to replace this UI with another one.   We know all these things  — and we know leaving that content, even if it’s abstracted into a string table or a resource bundle is going to come back to haunt us; no matter the abstraction: it’s part of the build, developers need to update it.  Developers and Systems folks need to deploy it.  

Smart developers separate code from content.  They make sure that the content in the application is completely independent of the build and deploy cycle of the application itself.  Where appropriate they make sure non-technical business users and have access to the externalized content and can update it and publish changes at any time.

Consider the following scenario:  Your application is for an insurance company.  Along comes major regulation change.  Now your (and every other application) needs to change its language accordingly.  If the content was separate from the application the response time and cost to make changes is minimal.  It’s business as usual.  If the content is not separate you are looking at months of builds, testing and deployment that costs time, opportunity and a tremendous amount of money,

What we’re really talking about is application architecture.  Making the right decisions has significant impacts on our development teams and process and as we’ve illustrated, our organization’s responsiveness and bottom lines.

Solving the Content Problem Through Application Architecture

This problem is very similar to another problem:  Websites.  Waaaaaaaaaaaaay back in the 1990’s people built and maintained entire websites in HTML by hand.  You had to be a programmer who understood the markup and how to update it and ultimately deploy it to a server once complete.  This model didn’t work. Once business started using the web to communicate to the masses marketing departments started getting involved.  Marketers wanted constant updates. Developers wanted to write code, not update copy.  They needed a solution.  The Web Content Management System (WCM for short) was born.  WCM offered a new architectural solution to manage websites. WCM allowed the developers to put the code into templates that contained placeholders for the content and gave authors a user interface to manage and edit the content at any time.  The content is managed separately from the code (the templates) and the two are brought together to create the final product. Everyone wins.

We need the same capabilities that WCM provides but for our applications.

Why Not Use a Traditional CMS?!

Simple.  Traditional CMS platforms have the wrong underlying design. Most CMS platforms were built or are based on the same design as CMS’s that were built 20 years ago.

  • They were built to manage pages:  Most CMSs do not have a generic concept of what content is.   These systems were built to manage pages.  You might be able to contort the system to get what you want from it but it will be a hack. You need a CMS that is agnostic to the kind or type of content you need to manage.
  • They rely on the wrong type of data tier: RDBMS/SQL databases were the primary backends for data regardless of the problem and open source options like MySQL were readily available.  You need a CMS that leverages a data tier that is more distributed and flexible.
  • They are built on the wrong technology: Many CMS platforms are built in PHP.  That’s fine for certain use cases. But frankly, Java has a larger community, more library support and a lot of powerful platforms like Solr, Elastic Search, Hadoop (and many others) that plug into a Java CMS perfectly.  Why run multiple tech stacks to accomplish a single goal?
  • They aren’t secure: Most CMS platforms couple authoring and delivery together in a single database.  That means work in progress (like pending policy updates) is available in the delivery runtime to anyone who finds a way into the database.  That can spell disaster in the event of a hack. Large companies and governments have suffered expensive and dangerous leaks due to this kind of coupled, insecure architecture.
  • They don’t support multi-tenancy: Most of the CMS platforms out there are not multi-tenant.  If you’re going to back apps with a CMS you need a CMS that will let you properly segregate and secure content between apps.  No one wants to do a CMS install PER app.
  • They don’t scale:  RDBMS is hard to scale.  You either cluster or replicate.  Both have serious issues when it comes to really large, global and auto-scale type deployments.  Supporting an app with your CMS will demand scale. Design for scale from the start.
  • They don’t fit into your development tools and process: When most CMS platforms were designed the content authors were the only audience that mattered.  Today it’s different.  Developer tasks like managing templates, CSS, JavaScript, content models and other artifacts need to be just as easy — and it needs to fit in with your existing workflow.  That means Git source code management, Continous Integration (CI), Integration with your IDE, the ability to fork and support multiple teams simultaneously.

If Not Traditional CMS, Then What? A Modern CMS, That’s What!

To get different results you need a different design.  You need a modern CMS:

  1. Built with the right technologies like Java, Spring, Groovy, Solr, Git.  These are the technologies that will be easy to hire for, will fit into and integrate with any enterprise and will have the out of the box capabilities and horsepower to back any scale deployment.
  2. Decoupled architecture that cleanly separates authoring and content delivery responsibilities on to separate, independently highly securable, highly scalable infrastructures by a publishing process.  Decoupled Architectures don’t have work in progress outside the firewall.  They are much easier to scale.
  3. Built on scalable document-oriented data tiers like XML+disk/Git and Casandra and others.  Disk-based tech is much better for streaming rich content and support global distribution better because they don’t need to be clustered.  At a minimum, you want something that’s document oriented and that distributes globally better than a than a SQL backend RDBMS.
  4. Content type agnostic:  From strings to web pages, to videos and virtual reality experiences.  The CMS needs to accept any kind of content.  That comes down to how content is modeled and how it’s stored.  Does the CMS store content in a SQL database or a JCR repo?  Red flag.  Can’t provide Headless Content / APIs AND rendered content?  Another Red Flag.
  5. Is Multi-Tenant. You need to be able to support many customers/apps on a single install. Either you support it or you don’t and today it’s a must!
  6. Supports authors AS WELL AS developers and DevOps. Developers don’t want to work in some clunky CMS.  They want to work locally in an IDE, then be able to work with their team and ultimately go through the entire workflow out to production.  A modern CMS gives them this.
  7. Open Source.  Open is better than closed. The debate is over. Is it a must?  No. But you’ll get more for your money and you’ll get a lot more innovation at a faster rate if you leverage open source.

Where can I find a Modern CMS?!

Check out Crafter CMS!  Crafter CMS is a modern, 100% open source Java based CMS built on Git!  It’s easy for authors, Amazing for developers and Awesome for DevOps!  Crafter doesn’t sit on a SQL database or a JCR repo like so many CMS platforms out there.  It’s built on Git, Java/Spring, and Solr.  It brings the power Git to CMS!  Authors have true versioning.  Developers and DevOps can work they way they are used to, with the tools they are used to based on the technologies they already know.  Crafter CMS is 100% content type agnostic, highly secure, high performance and scales like nothing else.  If you’re building any kind of application or digital experience, you have content, and you NEED Crafter CMS.  Great experiences are crafted!


Patterns for Integrating 3rd Party Content in Your CMS

There are often times when you will want to integrate content in your site that does not ORIGINATE in your content management system.  In this article we’ll look at a few high level patterns for integration and discuss the pros and cons of each approach.

Approach 1: Sync to CMS, publish to site as full content

Approach 1 focuses the integration at the CMS level only.  Content is either pulled or pushed from the 3rd party system in to the CMS.  From there it can be further processed and is ultimately published to the website where it can be used like any other type of content including native involvement in search and targeting scenarios.


  • 3rd party content is converted to 1st class content which allows it to be used natively by the system with not rendering engine level integration.
  • Authors may augment the content coming from the 3rd party system to include additional content and metadata
  • This integration is typically very simple
  • There is no impact on the performance or complexity profile of your delivery environment


  • Syncs can be tricky.  Remember content originates in the 3rd party system but it’s possible that it may need to be further modified by the CMS.  
  • A mechanism and a process must be defined that allows content updates to flow from the 3rd party system in to the CMS.  Typically this involves a strong ID that identifies each object, a mapping of fields that can always be overridden in the CMS  and mechanism for communicating creates, updates and deletes.

Other Considerations

  • By passing through the CMS 3rd party content is not “immediately” available on the website.  Content must wait for the sync mechanism to operate. Further, there may be workflow required before content can be made live.

When to use this pattern

  • This pattern is often preferred
    • It is relatively simple
    • Allows augmentation of the content in the CMS
    • Requires no run-time integration
    • Has no impact on the performance and SLA of the run-time

When not to use this pattern

  • When any create, update or delete to the content in the 3rd party system must be IMMEDIATELY available on the live site.

Approach 2: Partial sync to CMS, full details retrieved at run-time

Approach 2 syncs a simple jacket for the content to the CMS but relies on runtime integration with the 3rd party system for the full details of the 3rd party content.  The jacket in the CMS carries only the metadata required for the 3rd party content to participate in search, targeting and other dynamic behavior on the site.   The “sync” that creates the jacket in the CMS may be done through a simple integration or can even be a manual process.


  • Less content is relied on from the CMS which means that for items that have been previously published, detailed data is always 100% in sync with the 3rd party system.
  • Search, targeting and other dynamic capabilities native to the delivery system may still be relied on for functionality within the site.


  • There is a runtime dependency on the third party system for the actual content.  If that site is down, the content is unavailable.
  • As discussed in approach 1, syncs can be tricky.  In the case of approach 2 we still have some sort of manual or automated sync to create a jacket in the CMS.
  • In this approach there are 2 points of integration 1 at the CMS and one at delivery.  When you add additional moving parts, you are adding operation complexity that must be maintained and supported.
  • Content can be out of sync between what the site has in it’s jacket (and this in the indexes) and what is in the 3rd party system.  You must consider all cases: create, update, and delete

Other Considerations

  • By creating a run-time dependency between the delivery system and the 3rd party system you dictate that the 3rd party system must (generally) have the same reliability and performance capabilities of the delivery system.

When to use this pattern

  • When detailed aspects of the 3rd party content must be IMMEDIATELY up to date.
  • When it is impractical to import all of the 3rd party content in to the CMS.  A typical use case for this approach is 3rd party hosted video.  The CMS may contain only basic metadata and thumbnails while the 3rd party system retains all of the various video encodings.

When not to use this pattern

  • If the website and the 3rd party system must be 100% in sync on ALL aspects of the content at all times
  • When approach 1 is viable.

Approach 3: Run-time integration only

Approach 3 focuses integration at the content delivery tier only.  Display and behavior related to the 3rd party content is all handled through integration.


  • Approach is the most simple from a “moving parts” perspective. There is no syncing involved
  • 3rd party content is immediately up to date when changes are made in the 3rd party system


  • There is a runtime dependency on the third party system for the actual content.  If that site is down, the content is unavailable.
  • All content display, and behavior such as query driven listings, content targeting, and search must be integrated.
  • It can be difficult to integrate functionality like search where CMS content and 3rd party content are placed together on the page in a seamless way.  This is because the implementation is very likely to require issuing at least two queries (one to the CMS and one to the 3rd party system.)

Other Considerations

  • By creating a run-time dependency between the delivery system and the 3rd party system you dictate that the 3rd party system must (generally) have the same reliability and performance capabilities of the delivery system.

When to use this pattern

  • When 3rd party content must be IMMEDIATELY available and up to date on the site the instant it is updated in the 3rd party system.
  • When there is no need to rely on native capabilities of your delivery platform for delivery of the content and dynamic behavior (like search.)

When not to use this pattern

  • When option 1 or 2 are viable and use of native capabilities of the content delivery platform useful / required.