Five Reasons Why You Should Use a Git-based CMS (Part 4 of 5)

In our previous posts we looked at Crafter CMS and its Git-based versioning (part 1), distributed repository (part 2) and deployment mechanics along with its decoupled architecture (part 3)  In this post we’ll take a deeper dive into a feature of Crafter’s Git-based CMS that provides unparalleled support and speed for innovation: branching.

Imagine a single train track that stretches from Washington D.C. to New York.  How many trains can run simultaneously on this track?  How fast can the trains go? How much control over the order of arrival do you have?

  • You can run as many trains as you have room for on the track.
  • Anyone train can only go as fast as the train in front of it.
  • The trains always arrive in the order they departed in.
  • If a train breaks down or gets put on hold, it’s nearly impossible to reorder.

With only one track, the railroad’s ability to deliver success is throttled by the amount of volume they can handle and even when capacity is not an issue, they are completely dependent on good luck with respect to order and unplanned changes and breakdowns. With railroads, the solution to this problem is implemented with switches and sidings. Sidings are branches in that single track that allow a train to pull off the main line giving the controller the ability to regulate the order, speed, and quantity of trains on the main line.

The same problems of bandwidth, throughput, and order you see in the railroad example exist with projects that need to go through your traditional CMS.  Traditional CMS platforms have no ability to branch the content and code base. They are just like that single track from DC to New York.  This means you have very little control over the order, speed, and capacity of your development. Your CMS needs branching. Let’s explore this further.

Reason #4: Branching

Nearly every CMS still sits on an architecture and repository that was devised 20 years ago.  Back then innovation was important but the world moved a lot slower.  A single pipeline of features got the job done and the CMS was less of a bottleneck.  Today, in contrast, digital channels are at the core of many organizations’ strategy for serving the customer better and beating the competition.  The organization who moves the fastest often wins.  Agility is key.

Meanwhile, infrastructure costs have gone down while development costs continue to rise.  Balancing costs with volume and speed of innovation is a major challenge of our day.  Today it’s all about great DevOps. To make DevOps really work with CMS you need to be able to branch.

Traditional software development process has included branching for eons. Developers and DevOps have long since figured out that they need to be able to work in teams, isolate work and control the order in which work is merged into the critical path for go-live. The CMS track runs right alongside the traditional development track, and at some point, it merges and the last few steps require the CMS.  It’s only now that the demand for the speed of innovation has increased that the connective tissues between development and the CMS have been put under so much stress that they are completely failing.  The fact is that today, we need that same agility all the way through the CMS and right up to the very last step of production deployment.

Even if the majority of your development is outside the CMS you still need to integrate. Consider the following example:

Because the website needs to integrate with, and ultimately deliver the functionality of the microservice, we need to perform development.  We also need to support daily content edits and continuous publishing. With traditional CMS we have no option but to use multiple CMS environments to support this scenario.  Does this approach give us multiple tracks and control over my releases?  Yes, technically it does.  But practically speaking?  No.  Not at all.

As we learned before, moving content and code between CMS environments with traditional CMS architecture is extremely difficult. The process of spinning up environments and loading code and content into them is so difficult and time consuming for any DevOps team that most won’t even consider it unless absolutely forced to.  Even then the size of the team may not support the need. The rate of innovation crawls to a near stop.

To address this problem, we built Crafter CMS v3 on top of a Git-based repository. As a result, the Crafter CMS platform is built on repository store that not only branches but also is fully distributed.  Not only are you able to easily control the order and rate of work, but it’s a snap to move work from one environment to another.

Moreover, branching not only supports DevOps and but it makes development easier.  Developers and authors can experiment, work on major features and other site enhancements in the safety of branch-based sandboxes that keep them from stepping on each other’s toes.

Conclusion

If you want to quickly innovate with your website, mobile app and other content-rich digital experience apps, you will need multiple teams working on different features at the same time. You need control of who is working on what and the order in which projects will be delivered.  Having the capability to manage these concerns with agility is the key to innovating quickly.  Traditional CMS platforms don’t support the basic feature set that enables this. Moving most of the development outside the CMS only gets you so far. You need a CMS that supports your DevOps process with features like branching and distributed repository if you truly want to be able to move fast.

Stay tuned for our next blog entry to learn another major reason why you should use a Git-based CMS!

Match Highlighting for Search in Crafter CMS

Highlighting search terms in search results is a common requirement for many websites.  Crafter CMS builds on top of Apache Solr and make implementing rich search and other query-driven experiences super simple.

In this tutorial, we’ll create a simple article search backend that highlights the search terms that were used within the results returned to the user.

Step 0: Prerequisites

If you haven’t gotten Crafter CMS set up and built your first site you can follow this tutorial to get started: Working with Your First Crafter CMS Web site.

Step 1: Build a content model for articles

The next thing we need is content to query against.  The first step in supporting content creation is defining the Content Type.  The Content type is the definition of the structure of a particular type of content.  In our example, we want to define the structure of an Article.

A simple article should have:

  • A Url
  • A title
  • An author name
  • A body

To define this we use Crafter’s Content Type management console:  Below you can see the example model including the fields and their types.

You can learn more about content modeling here Content Modeling in Crafter CMS.

Step 2: Create content

Now that you have your article content type defined you can create articles.  To create an article open the pages folder (we modeled the article as a page with a URL) in the sidebar and right click on the home page:

Step 3: Create a REST script to return highlighted results

Now that you have content you can write a RESTful controller and test it.  Let’s create a simple GET based REST controller.

  1. Open the Sidebar and locate the Scripts folder.
  2. Open the Scripts folder and navigate to “rest” folder.
  3. Right click on it and choose “Create Controler
  4. In the script name dialog, enter “search.get” and click Create.
  5. Enter the following code:
// build a query
def keyword = params.q 
def queryStatement = "content-type:\"/page/article\" " 
 
 if(keyword) {
      queryStatement += " AND $keyword"
 }
 
 def query = searchService.createQuery()
 query.setQuery(queryStatement)
 query.addParam("hl", "true")
 query.addParam("hl.fl", "body_html")
 query.addParam("hl.simple.pre", "<b>")
 query.addParam("hl.simple.post", "</b>")
 
 // execute the query
 def executedQuery = searchService.search(query)

def matches = [:]
matches.found = executedQuery.response.numFound
matches.articles = executedQuery.response.documents
matches.highlights = executedQuery.highlighting

return matches

Step 4: Execute the Script

In a browser, go to http://SERVER:PORT/api/search.json?q=AWORDTHATEXISTSINRESULTS

example: http://localhost:8080/api/search.json?q=bacon

Work Your Own Way with Crafter CMS (Series Part 1): Step-through Debugging

Most CMS platforms do a decent job of simplifying content and digital experience creation and editing for non-technical content managers.  The challenges really start once you need to innovate and development is required.  Traditionally CMS platforms have been pretty bad for developers.  They require a lot of CMS specific knowledge and don’t integrate with developer tools and process.
Here are 7 things that developers really want with a CMS:

  1. Let me work locally with my own tools like my IDE and my source code management
  2. Let me leverage my existing skills.  I want a low learning curve. Don’t make me learn a new, niche framework
  3. Let me work in teams on multiple projects at the same time without interfering
  4. Let me maintain a real development process
  5. Make the integration with the CMS seamless
  6. Don’t make me do builds
  7. Don’t make me do heavy deployments

In this installment of the Work Your Way Series we’re going to tackle item #1 (Let me work locally with my own tools like my IDE and my source code management.)   Let’s start with some background: Crafter CMS uses Git as its primary content store.  That’s the foundation of the solution for developer desire #1. A developer can mount a local clone of a Crafter CMS project directly with their IntelliJ, Netbeans, Eclipse or other IDE.  That means they can use their preferred development tools to edit and debug code and templates.  And as they work, all of the changes they make are tracked by the Crafter CMS via its native Git support.  Sounds awesome right?  Let’s learn how to get set up:

Step 1: Get a local copy of Crafter CMS running

You are going to use your IDE to update and debug your code and templates.  You’ll want a local instance of Crafter CMS running so you to execute the code and attach your debugger to.

To install and get Crafter CMS running (authoring environment) locally follow this guide:
http://docs.craftercms.org/en/3.0/getting-started/quick-start-guide.html

Step 2: Start a Crafter CMS project

Now that Crafter CMS is up and running let’s create a project so we have a place to work.  We’ll create a local project for the sake of this article, however, as we’ll learn later in this series, it’s possible to clone a remotely managed project as well.

To create your CMS project follow this guide:
http://docs.craftercms.org/en/3.0/getting-started/your-first-website.html

Step 3: Mount your IDE on top of your Crafter CMS project

For this article, I’m going to use IntelliJ.  Any IDE that allows you to connect to a remote JVM via Java Debug Wire Protocol (JDWP) should work.  Eclipse and Netbeans are other solid IDE options.

  1. Within IntelliJ, create a new project:

  • Select Create New Project

 

  • Choose Groovy

  • A: Configure your project name to something you like
  • B: Browse to the Git repo “Sandbox” under your “CRAFTER-INSTALL/data/repos/sites/PROJECTID/sandbox”
  • C: Change the module name from Sandbox to your project name

 

  • Once your project is created, open it to see all the files.  IntelliJ automatically recognizes you are sitting on top of a Git repo and will allow you to do Git operations right from the IDE.
  • Note that a Groovy project creates an “src” folder.  You can delete this.  You don’t need it.

  • Right click on your model and add a new file called “.gitignore”
  • Do not add this file to Git when prompted.

The contents of the .gitingore file should be as follows:

*.iml
.gitignore
.idea/*
  • This will tell Git and Crafter CMS to ignore your IDE project files

  • If you were to run a “git status” command in your project sandbox you would note that Git is totally satisfied and ready to rock 🙂

 

Step 4: Set up step-through debugging

Great, now that we’ve got our IDE sitting on top of our project and we’ve got the Git integration configured we’re ready to set up the step through-debugging.

  • Shut down Crafter CMS
  • Add the following lines to CRAFTER-INSTALL/bin/apache-tomcat/bin/setenv.sh (or setenv.bat)
    • Syntax in .bat will be slighly different

export CATALINA_OPTS="$CATALINA_OPTS -agentlib:jdwp=transport=dt_socket,address=localhost:8000,server=y,suspend=n"
  • Start Crafter CMS

Back in our IDE we can now configure and start our remote debugging session!

  • Go to “Run” and select “Debug”

  • Since this is a new project we’ve got to configure our debugging connection.  The next time you click Debug you’ll just select the existing configuration and click Run.

  • Click the + (plus) icon to create a new configuration
  • Select “Remote”

  • Give the debug configuration a name
  • Change the port from 5005 to 8000 (to match the value in setenv.sh)
  • Click “Apply” to accept the changes
  • Click “Run” to attach to the JVM running Crafter CMS

  • Congrats! If your JVM configuration is correct and your ports match in the debug config when you click “Run” you’ll see the “Connected” message in the console as shown above.

Step 5: Start debugging

Now that we’re connected we can start stepping through code in our Groovy Scripts.

  • Open the “Scripts” folder in your project
  • Browse down through classes, org, craftercms, sites, editorial and open SearchHelper.groovy
  • Set a breakpoint at line 56.  This method is used by the Home Page controller of the editorial website we created in step 2.
  • In a browser, open the Crafter CMS preview for the homepage of the website we created

  • When you load the homepage in Crafter Studio the rendering will pause

  • IntelliJ’s window will typically come to the foreground automatically.
  • The thread has been paused and the IDE is now in control of the thread execution.
  • You can now use the step-through debugging tools in the IDE to walk through the code.

Step 6: Magic

Found a bug?  Here’s where things get really fun.  We’re about to see the benefit of bringing Git, Groovy and your IDE together in one place.  Fixing the bug and sharing that fix with the rest of your team is a breeze:

  1. Once you understand the bug and you know what code you want to change, click Play to let the thread complete.
  2. Using the IDE, fix the Groovy code.
  3. Simply refresh the browser again to test.  Step through and verify things are working.
  4. Using the IDE to interface with Git, commit the code and push the code forward in your team mates.
  5. Find something fun to do or work on.  You have a lot more time on your hands now that you are working in a truly integrated, no-compile required, easy to code and debug CMS 🙂

 

 

Working with Content as XML/DOM in Crafter CMS

In a previous article (Querying Content in Crafter CMS) we talked about how you can query content in Crafter using content and search services. Under the hood, Crafter CMS saves all of the content you create through the authoring interface (Crafter Studio) as XML.  This XML is published from Crafter Studio to dynamic delivery engines (Crafter Engine) and is available through the content services and is also indexed in Solr and thus available through Crafter’s search services.  There are times in the rendering engine (Crafter Engine) when you may want to get access to values in the XML directly and work with the content as an XML Document Object Model (DOM) API.

In the example below I will show you how you can:

  1. Load a content object
  2. Get access to its DOM
  3. Query for DOM elements within the parent object
  4. Process the results

To illustrate this let’s imagine that we have a use case where content authors need to manage values like CEO name or Number of Offices throughout a site that might change.  They want to pepper these values throughout the site but want to update them quickly from a single place.    To do this we would use a macro/placeholder type approach in the content where the author would enter content with macro/placeholder in the place where the value should go.

Example:

Acme Anvil Co has [Office_count] offices throughout the world.

To manage the Macro/Value pairs we’ll give the authors a content type:

Once you have your macro type defined you can create a content object based on that type:

Now that we have created an author managed list of macros and their corresponding values.  Let’s create a basic controller (some assumptions to keep things simple) to process the content in a component and replace the macros found in the content with the value.

Controller example:

// Load the content object containing the macros key values
def macroItem = siteItemService.getSiteItem("/site/components/5ef8d326-3e85-9f67-e8d6-47fb5dfba21d.xml")

// Get the unprocessed content
// Here we'll put some assumptions in the code.  In reality you may want 
// to process all the content fields in the content.
def content = contentModel.queryValue("content_html")
def contentUpdate = content
// Query the macros and iterate over them performing the replace. 
// this is where we get the DOM and leverage the DOM API
def macros = macroItem.getDom().selectNodes("//item")
macros.each { pair ->
 def macro = pair.valueOf("./macro")
 def value = pair.valueOf("./value")
 contentUpdate = contentUpdate.replace("["+macro+"]", value)
}

// Make the processed content available to the template
templateModel.content_html = contentUpdate

 

You can easily tie this controller into any template by adding it to the top of the template:

<@controller path="/scripts/controllers/macro-controller.groovy" />

Most of the time with Crafter CMS, there is no reason to access the DOM directly.  From time to time you may find it more convenient or necessary.  Now you know how!

Why Developers Should Care About CMS

As developers, we’ve got a strong handle on how to manage and deploy our code assets.  Yet every one of us, at some point in our application build has said, “What about this text? What about these images? Where do these belong?”  That’s pretty universal.  Nearly every single application today has content in it.  Be it a web app or a native app; it’s full of strings, images, icons, media and other classes of content.

This content doesn’t really belong in our code base — because it’s not code.  These non-code assets make us as developers pretty uneasy.  We know that at some point a business user is going to ask us to make a change to one of those strings and we’re going to spend hours of build and deploy cycles to handle a 30-second code change.  We know that at some point we’re going to need to translate that content.  We know at some point we’re going to replace this UI with another one.   We know all these things  — and we know leaving that content, even if it’s abstracted into a string table or a resource bundle is going to come back to haunt us; no matter the abstraction: it’s part of the build, developers need to update it.  Developers and Systems folks need to deploy it.  

Smart developers separate code from content.  They make sure that the content in the application is completely independent of the build and deploy cycle of the application itself.  Where appropriate they make sure non-technical business users and have access to the externalized content and can update it and publish changes at any time.

Consider the following scenario:  Your application is for an insurance company.  Along comes major regulation change.  Now your (and every other application) needs to change its language accordingly.  If the content was separate from the application the response time and cost to make changes is minimal.  It’s business as usual.  If the content is not separate you are looking at months of builds, testing and deployment that costs time, opportunity and a tremendous amount of money,

What we’re really talking about is application architecture.  Making the right decisions has significant impacts on our development teams and process and as we’ve illustrated, our organization’s responsiveness and bottom lines.

Solving the Content Problem Through Application Architecture

This problem is very similar to another problem:  Websites.  Waaaaaaaaaaaaay back in the 1990’s people built and maintained entire websites in HTML by hand.  You had to be a programmer who understood the markup and how to update it and ultimately deploy it to a server once complete.  This model didn’t work. Once business started using the web to communicate to the masses marketing departments started getting involved.  Marketers wanted constant updates. Developers wanted to write code, not update copy.  They needed a solution.  The Web Content Management System (WCM for short) was born.  WCM offered a new architectural solution to manage websites. WCM allowed the developers to put the code into templates that contained placeholders for the content and gave authors a user interface to manage and edit the content at any time.  The content is managed separately from the code (the templates) and the two are brought together to create the final product. Everyone wins.

We need the same capabilities that WCM provides but for our applications.

Why Not Use a Traditional CMS?!

Simple.  Traditional CMS platforms have the wrong underlying design. Most CMS platforms were built or are based on the same design as CMS’s that were built 20 years ago.

  • They were built to manage pages:  Most CMSs do not have a generic concept of what content is.   These systems were built to manage pages.  You might be able to contort the system to get what you want from it but it will be a hack. You need a CMS that is agnostic to the kind or type of content you need to manage.
  • They rely on the wrong type of data tier: RDBMS/SQL databases were the primary backends for data regardless of the problem and open source options like MySQL were readily available.  You need a CMS that leverages a data tier that is more distributed and flexible.
  • They are built on the wrong technology: Many CMS platforms are built in PHP.  That’s fine for certain use cases. But frankly, Java has a larger community, more library support and a lot of powerful platforms like Solr, Elastic Search, Hadoop (and many others) that plug into a Java CMS perfectly.  Why run multiple tech stacks to accomplish a single goal?
  • They aren’t secure: Most CMS platforms couple authoring and delivery together in a single database.  That means work in progress (like pending policy updates) is available in the delivery runtime to anyone who finds a way into the database.  That can spell disaster in the event of a hack. Large companies and governments have suffered expensive and dangerous leaks due to this kind of coupled, insecure architecture.
  • They don’t support multi-tenancy: Most of the CMS platforms out there are not multi-tenant.  If you’re going to back apps with a CMS you need a CMS that will let you properly segregate and secure content between apps.  No one wants to do a CMS install PER app.
  • They don’t scale:  RDBMS is hard to scale.  You either cluster or replicate.  Both have serious issues when it comes to really large, global and auto-scale type deployments.  Supporting an app with your CMS will demand scale. Design for scale from the start.
  • They don’t fit into your development tools and process: When most CMS platforms were designed the content authors were the only audience that mattered.  Today it’s different.  Developer tasks like managing templates, CSS, JavaScript, content models and other artifacts need to be just as easy — and it needs to fit in with your existing workflow.  That means Git source code management, Continous Integration (CI), Integration with your IDE, the ability to fork and support multiple teams simultaneously.

If Not Traditional CMS, Then What? A Modern CMS, That’s What!

To get different results you need a different design.  You need a modern CMS:

  1. Built with the right technologies like Java, Spring, Groovy, Solr, Git.  These are the technologies that will be easy to hire for, will fit into and integrate with any enterprise and will have the out of the box capabilities and horsepower to back any scale deployment.
  2. Decoupled architecture that cleanly separates authoring and content delivery responsibilities on to separate, independently highly securable, highly scalable infrastructures by a publishing process.  Decoupled Architectures don’t have work in progress outside the firewall.  They are much easier to scale.
  3. Built on scalable document-oriented data tiers like XML+disk/Git and Casandra and others.  Disk-based tech is much better for streaming rich content and support global distribution better because they don’t need to be clustered.  At a minimum, you want something that’s document oriented and that distributes globally better than a than a SQL backend RDBMS.
  4. Content type agnostic:  From strings to web pages, to videos and virtual reality experiences.  The CMS needs to accept any kind of content.  That comes down to how content is modeled and how it’s stored.  Does the CMS store content in a SQL database or a JCR repo?  Red flag.  Can’t provide Headless Content / APIs AND rendered content?  Another Red Flag.
  5. Is Multi-Tenant. You need to be able to support many customers/apps on a single install. Either you support it or you don’t and today it’s a must!
  6. Supports authors AS WELL AS developers and DevOps. Developers don’t want to work in some clunky CMS.  They want to work locally in an IDE, then be able to work with their team and ultimately go through the entire workflow out to production.  A modern CMS gives them this.
  7. Open Source.  Open is better than closed. The debate is over. Is it a must?  No. But you’ll get more for your money and you’ll get a lot more innovation at a faster rate if you leverage open source.

Where can I find a Modern CMS?!

Check out Crafter CMS!  Crafter CMS is a modern, 100% open source Java based CMS built on Git!  It’s easy for authors, Amazing for developers and Awesome for DevOps!  Crafter doesn’t sit on a SQL database or a JCR repo like so many CMS platforms out there.  It’s built on Git, Java/Spring, and Solr.  It brings the power Git to CMS!  Authors have true versioning.  Developers and DevOps can work they way they are used to, with the tools they are used to based on the technologies they already know.  Crafter CMS is 100% content type agnostic, highly secure, high performance and scales like nothing else.  If you’re building any kind of application or digital experience, you have content, and you NEED Crafter CMS.  Great experiences are crafted!

 

Active Cache a RESTful Response in Crafter CMS

This article will demonstrate how to create a RESTful service in Crafter Engine that has predictable performance and reliability when your service relies on an external service.

Any time your services depend on another service there is a cause for concern. You can’t control the performance or the availability of the external service. Further, if the response of the external service is not unique across calls then there may be no real need to call out to it on each request you receive.

In this case, what you want to do is cache the request from the external service and have your service attempt to get the content from the cache. Active cache is a built in Crafter CMS capability that makes building these sort of solutions much easier. You tell Active Cache what you want, how to get it and how often to refresh it in the background. From there on, you simply ask Active cache for whatever the current response is.

Step 1: Create a REST Controller

Under Scripts/rest right click and click create controller
Enter my-data.get as the controller name
Add the following code to the controller.

 import org.craftercms.core.cache.CacheLoader
 import org.craftercms.core.service.CachingOptions
 import groovy.json.JsonSlurper

 def cacheService = applicationContext["crafter.cacheService"]
 def cacheContext = siteContext.getContext()
 def myCacheKey = "aServiceCallResponse"
 def loader = new ExternalServiceLoader()

 def value = ""
 def responseItem = cacheService.get(cacheContext, myCacheKey)

 if(responseItem == null) {
    // item is not cached, get the value
    def myObject = loader.load()
    value = myObject

   // cache the value with a loader to periodically refresh its value
    def cachingOptions = CachingOptions.DEFAULT_CACHING_OPTIONS
    // def cachingOptions = new CachingOptions() // define your own options
    // cachingOptions.setRefreshFrequency(1) // set the number of ticks for refresh

    try {
        cacheService.put(cacheContext, myCacheKey, myObject, [], cachingOptions, loader)
    }
    catch(err) {
        logger.error("error adding ${myCacheKey} to cache: ${err}")
     }
 }
 else {
    value = responseItem
 }

 return value

/**
 * Define an active cache loader that will be used to prime and then
 * periodically refresh the cache with the latest data from an external
 * service.
 */
 class ExternalServiceLoader implements CacheLoader {
    Object load(Object... parameters) throws Exception {
       def externalServiceHost = "http://api.population.io/1.0"
       def externalServiceURL = "/population/United%20States/today-and-tomorrow/"

       // call the service
       def response = (externalServiceHost+externalServiceURL).toURL().getText()

       // parse service's the JSON response to an object
       def result = new JsonSlurper().parseText( response )

       return result
    }
 }

Step 2: Execute the Service

Open a browser and hit the following URL: http://localhost:8080/api/1/services/my-data.json
Note that your hostname, ports and pageId values may differ than the example
See results

Building CMS Backed Augmented Reality Apps with Crafter CMS

From SnapChat to Pokemon Go to the helmets used by F-35 fighter pilots, Augmented Reality (AR) applications are gaining a ton of traction. Unlike virtual reality which attempts to completely replace actual reality with an alternative universe and sensory experience, augmented reality, leverages technology to add a sensory (typically visual) layer of interaction and data on top of our everyday experiences. You can learn more about the difference between VR and AR by watching this free tech-talk / webinar. Augmented reality is so exciting and opens up so many new opportunities because it enhances our physical experience, giving us a way to experience things in a way we never could before.

As with most applications today, AR applications frequently contain a lot of content and related digital media such as 3D models, images, and videos.  Often times textual/markup content and digital media are deployed statically along with the application. This approach creates unwanted dependencies for content updates on the developers and requires laborious deployment processes.

In a business context, this misalignment and lack of independence can slow activities way down and ultimately impact the bottom line.  To address this common problem, developers often build custom applications for non-technical users or leverage content management system (CMS) technology.  Building custom authoring tools double development time and creates an additional support burden on your development team. Leveraging a CMS is a great idea, but you need to make the right choice.  Many CMS architectures impose severe limitations on markup, application and API architecture and design. As we’ll see in this article, Crafter CMS provides you with the tools and flexibility you need to create innovative, rich experiences like Augmented Reality.

In this article, we’ll build a simple (but not too trivial) AR application. After that, we’ll update it so that non-technical users can upload new models and edit and deploy/publish changes to the scene through Crafter CMS without any technical background.

For our application, we’ll start with a simple AR application based on open source frameworks AR.js, ARToolkit, and THREE.js.  Given that this is a tutorial and we want to have some fun with it, we’ll use (USC 17 107) some freely available 3D Models from (http://roestudios.co.uk/project/3d-pokemon-models.) You may also use any other models as long as they can be loaded by THREE.js’s library of loaders.  Free models vary in availability and quality. I chose this library because of the number of available models.

 

Step 1: Get a Simple, Static Browser-based Augmented Reality Application Running

Before we put anything in the CMS, let’s first make a basic static AR example application work.  We won’t get into the nuts and bolts of the various AR and 3D web related libraries.  There is a lot of material and examples already available on the subject.  Instead, we’ll try to avoid a lot of abstraction and construct an example that is simple enough in what it does so that the code can be readily understood.  You can find the code here on my Github account.

  • Clone or download this code as a zip
  • Open code/simple-ar.html with a browser

In your browser, you should see a video of yourself (captured by  your camera if it’s available and you have authorized it to be used by the browser.)  Additionally, hovering on screen, there should be a 3D model of Pikachu.

Now that we have a VERY SIMPLE working example of an AR application working we can get started integrating content management capabilities with our CMS so that anyone can add new 3D models to the application.

Step 2: Get Crafter CMS Downloaded and Installed

The first step is to get the CMS downloaded and installed. Crafter CMS is free and open source software licensed under GPL v3.

2.1 Download the Crafter CMS bundle

2.2 Install Crafter CMS and start it up

  • Unzip the download
  • Execute startup.sh (or startup.bat if windows) inside the unzipped folder
    • Prerequisite: Java 8 JDK is installed, and JAVA_HOME is defined as an environment variable
  • Open a browser to http://localhost:8080/studio. I recommend using Chrome. Chrome has the best support for desktop file drag and drop of all the browsers which will come into play later in this article.studio-login
  • Login with username ‘admin‘ and password ‘admin.’studio-no-sites

Step 3: Install the Application as a Project in the CMS

We now have the prerequisites done, and we’re ready to party! Let’s install the application into the CMS.

3.1 Create and setup a CMS project to manage your application

  • In Step 1.2 you logged into the CMS as the admin user who has an admin role. On the screen, you will see a button on the left hand that says “Create Site.” Locate and click this button.

  • Enter AR Tutorial as the application name.  Crafter CMS will generate a Site ID for you.
  • Select “Empty” for your blueprint and click the Create button.  Crafter CMS will create a new project for your AR Tutorial application with a basic “Hello World” type configuration.hello-site

3.2 Upload your application to the CMS

  • Click on Site Content in the menu at the top of the screen. This will open a panel on the left
  • Click on the Static Assets label this will open the folder.
  • Right click on the static-assets folder and choose Bulk Upload Assetsupload-static-assets
  • In your code folder from step 2, you will find a static-assets folder.  Drag the contents of this folder onto the upload dialog in the browser. When it’s done, you will be able to click Done.
  • By uploading the compiled assets, we’ve loaded the app into the CMS.  Now we have to tell the CMS to render our single page.  To do this, we must update our out-of-the-box template provided to us by the blueprint.  In Crafter CMS, templates are coded in Freemarker and can contain any kind of markup (typically HTML 5) as well as Freemarker Directives that make it dynamic.

    3.3 Update the site template with the content from the HTML page

  • To update the template: click “Site Content” in the menu at the top to hide the Site Content panel.   Then locate the Wrench in the upper right-hand corner on the toolbar.  Click the Wrench.  This will open the Preview Tools panel on the right-hand side of the screen.  preview-tools
  • Now click on In-context Editing to expand that specific tool.  You will notice this tool allows you to edit content, templates, and controllers.  We’re interested in the templates.  Click Edit Template.edit-tempate
  • Replace the code in this template with the HTML found in your build directory at code/simple-ar.html.
  • Note that the static code example assumes the application assets are at ./static-assets on the server.  In Crafter CMS, static assets like CSS, Java Scripts and images are served from /static-assets.  You will want to modify the paths (remove the ‘.’) in the HTML template to use the /static-assets folder as shown above.  Once done, click update.
  • Your app is now running in the CMS!  Congrats!   With this complete, we can start working on making the application dynamic, editable by a non-technical user and updatable at any time without a deployment.

Step 4: Content enable the application

The first step in content enabling the application is determining what the content model is.  A content model is a definition and structure of the content.  We maintain the definition of the content structures within Crafter CMS by declaring Content Types.

To figure out what your Content Types should be, look for the NOUNS in the application that you want a non-technical user to be able TO easily update and publish at any time.  In our case, this is an individual 3D model of a Pokemon like Pikachu.

While the model in this simple application will be very simple, Crafter CMS is capable of modeling very sophisticated objects as well as relationships between objects to handle real world scenarios.

Before getting into the CMS, let’s figure out what the definition for our Pokemon type should be.  We do this by looking at the application and seeing what fields the app will require. Because this particular type of content is a 3D asset rather than traditional digital media, we need to understand a little about what goes into making a model and how it’s described. We’ll start by looking at the two models in the model directory.  By examining both the pikachu and snorlax model directories we can see that a model can contain a different number of files.  Complex models may contain a lot of files.

We want the act of adding these models to the CMS to be really easy.  Other than adding the files to the CMS there is nothing to configure about them. Further, a folder is a good grouping mechanism for keeping the files for a specific model together. Given this, we’ll assume that the first step for the author in adding a new model is to simply upload it by dragging the model folder into the static-assets model directory via bulk upload.  In other words, for the 3D models themselves, we’ll leverage folder structure and IA rather than Content Model to store and organize the raw 3D content assets.

The next thing to look at is how the model is loaded into, positioned and scaled in the world.  For this, we need to look at the code.  Specifically, the code that loads the 3D model:

By examining this we see there are a number of important attributes:

  • We also see that every model has a file that defines its structure (.obj)
  • And a file that defines how to apply its textures (.mtl)
  • The Model Scale Multiplier
  • The Object Rotation on the X axis
  • The Object Position on the X axis
  • The Object Position on the Y axis

If we think about it, some of these are directly related to the 3D model itself and belong to the Pokemon content model.  Other attributes like scale and position are used to place the model in the world.  It’s more flexible if we manage the scale and positioning attributes outside of the object.

OK! We have a list of fields we need for our Pokemon, let’s get started defining the Content Type in the CMS:

4.1 Declare a Pokemon 3D Model Content Type

  • Open the Site Content panel.
  • Locate and click the menu item for Admin Console. This will take you to the power user area in the CMS used to manage and perform various site administration tasks.
  • On the left panel in the Admin Console, locate and click the menu option for Content Types. This tool is used to create new Content Types and edit existing ones.
  • content-types-consoleOn the toolbar at the top of your screen, you will notice two options: Open Existing Type and Create New Type. Because a Pokemon is a new concept you want to introduce and define in the CMS, click on Create New Type. This action will pop up a dialog to prompt you for some basic information about the content type.
  • Enter Pokemon for the display label. Crafter CMS will suggest a content type name automatically for you.
  • The Type you want to choose is Component. In Crafter CMS anything that you are modeling which is NOT a page is a Component. The main difference in the CMS between pages and components is that page objects are automatically URL addressable while components are not.create-type-dialog
  • Once the dialog is configured, click Create. This will open the content type editor for your Pokemon Content Type. You can learn more about the content type editor in detail at http://docs.craftercms.org/en/stable/developers/content-modeling.htmlnew-pokemon-type
  • The Content Type Editor lets you drag/drop editing field types from the right-hand controls list into your Content Type definition. You then click on the individual controls and configure them with Labels, properties, data sources, and constraints.
  • For each field, you need to add (Object File, Texture file, etc.), look through available controls list on the right, find the appropriate editing control for that content and drag/drop it into the Pokemon properties area.
  • After you add a field to the Content Type, click on it to configure it in the Properties Explorer.
    • Add a Title. This is the user-facing name the author will see alongside the input control.
    • Crafter CMS will automatically suggest a variable name which you can use later in templates and controllers to access the content values.
    • Scroll down in the Properties Explorer to see all the properties and constraints for the given field. This is how you mark fields as required, change their input length and make other adjustments.
  • Because you want the user to be able to pick the object and texture fields, you will need to give them a file picker control.  In Crafter CMS this is called the Item Selector control. Drag one of these on the screen as shown above and give it a title of “Object File.”
  • Now you need to tell the Item Selector where to find the content it’s going to allow the user to pick from (remember the authors will upload models into the /static-assets/model folder via drag and drop.)  To wire this up, you will need a Data Source. In Crafter CMS the way picker type controls get their values is via a Data Source.  There are several out of the box Data Sources, and it’s possible to define your own as well.  For our example, because we want to pick an existing file in the repo we’re going to use the data source called File Browse.  Scroll down within the Data Sources area until you see it then drag/drop it onto the Data Sources portion of the Content Type definition.

    Click on the added Data Source and begin to configure it as shown above.  Note that we’re simply identifying a path elsewhere in the CMS where we want to pick a file from.
  • Once our Model Files Data Source is defined, let’s connect it to our Item Selector control for Object Files.  Click on the Object Files control and configure it in the property editor as shown below:
  • Add another field for Text Files and connect it to the same data source.
  • Add an input control for X Rotation and configure it as shown below:
  • Perfect! Your Content Type definition is now complete.  Click Save at the bottom of the screen.Note that you are going to get a warning that the content type does not have a template associated with it. That’s ok.  Not every Content Type needs a template.  In our case, the app already knows how to render the content, and we don’t need another template.  Click Save on the dialog.  You’ll notice a small save indicator on successful save.no-template-warn

4.2 Create Pokemon Content Objects

Now it’s time to create some test content.  To return to your project Click the Crafter Logo on the toolbar in the top left corner of the screen.  This takes you to your workflow dashboard.  From here you can create content or navigate back to your application preview via the Site Content panel.

  • Click on Site Content in the menu at the top of the screen. This will open a panel on the left
  • Click on the Components label this will open the folder.
  • Right click on components folder and Create Folder.

  • Provide the folder name “pokemon”.

  • By clicking Create, you will have added a pokemon folder to the components folder.  This is simply to help organize your content.
  • Right click on the pokemon folder and Create Content.
  • The CMS will prompt you for the type of content you want to create.  You will note the Pokemon option.  Choose Pokemon and click “Ok.”choose-conent-type
  • Once you click OK, you will be given the entry form for your Pokemon which you can start filling out.  Awesome! Crafter CMS automatically creates your Pokemon editing interface based on the Content Type Definition.

  • Repeat this process as many times as you want, creating a new Pokemon each time.  When you save the form, you will see the Pokemon item listed under the pokemon folder as whatever value you entered into the Internal Name field (which is only used by the CMS editing tools as a label in the tool.)
    • Note:  Included an X Access Rotation in the model object because based on the available models the models all start facing in different positions.  The same is true for scale.  You may wish to add a “normalizing” scale value in addition to the rotation.

Step 5: Configure the AR Page Controller to Show a Pokemon Loaded from the CMS

Now that you have content objects in the system it’s time to provide the CMS with the logic to determine which items to show.  To start, we’ll keep things simple.  We’ll provide the template with a static list of models to display.

5.1 Update the Controller

The controller contains the business logic.  In our case, the controller is for a “Page.”  In Crafter CMS you can have controllers for pages, components, services, jobs and other kinds of objects as well.  To get to the page controller:

  • Locate the Wrench in the upper right-hand corner on the toolbar.  Click the Wrench.  This will open the Preview Tools panel on the right-hand side of the screen.
  • Now click In-Context Editing to open the In-Context Editing tools.
  • Now click Edit Controller.

  • Now add the following code:

def pokemonModelIds = [ “/site/components/pokemon/snorlax.xml”]

templateModel.pokemonInstances = []

pokemonModelIds.each { id ->
def pokemonInstance = [:]
def modelItemId = id

pokemonInstance.model = siteItemService.getSiteItem(modelItemId)
pokemonInstance.xPosition = -2
pokemonInstance.yPosition = -2
pokemonInstance.zPosition = -2
pokemonInstance.scaleMultiplier = 1

templateModel.pokemonInstances.add(pokemonInstance)
}

  • Now click update to save the controller.

The changes we made to the controller have added a new variable called pokemonInstances to our template model.

5.2 Update the Template

Now that our template has pokemonInstances available to it, we have to update it to take advantage of the values in that data structure.  We’re going to iterate over the items in the array and call the addModelToScene function for each item.

  • Locate the Wrench in the upper right-hand corner on the toolbar.  Click the Wrench.  This will open the Preview Tools panel on the right-hand side of the screen.
  • Now click In-Context Editing to open the In-Context Editing tools.
  • Now click Edit Template.
  • Find line 99 and make the following updates

<#list pokemonInstances as pokemonInstance>

<#assign texturePath = pokemonInstance.model.textureFile.item[0].key />
<#assign objectPath = pokemonInstance.model.objectFile.item[0].key />

loadModelIntoScene({
basePath: “${texturePath?substring(0, texturePath?last_index_of(‘/’)+1)}”,
mtlPath: “${texturePath?substring(texturePath?last_index_of(‘/’) + 1)}”,
objPath: “${objectPath}”,
scaleMultiplier: ${pokemonInstance.scaleMultiplier},
xPosition: ${pokemonInstance.xPosition},
yPosition: ${pokemonInstance.yPosition},
xRotation: ${pokemonInstance.model.xRotation}}, scene);
</#list>

  • Now click update to save the controller.

AWESOME! Our template now renders models that we add through the CMS.

The only issue is, adding a new model to the scene requires us to edit the controller.  This is the job of a developer.  In essence, we’ve moved the responsibility of deciding what models to render from the Javascript to the groovy code, but we haven’t put the power of deciding what to show in the hands of the authors yet.  So let’s do that.

Step 6. Give the Author the Ability to Add Models to the Scene

6.1 Update the Scene Content Model

To give the author, a non-technical user, the ability to update the scene, we have to first update our content model.  In this project, the content model that controls the scene is called Entry.  To edit the Entry Content Model:

  • Open the Site Content panel.
  • Locate and click the menu item for Admin Console. This will take you to power user area in the CMS used to manage and perform various site administration tasks.
  • On the left panel in the Admin Console, locate and click the menu option for Content Types.
  • Now click Open Existing Types and select Entry.

  • Click the Body Content Field.
  • Now click the dark X in the upper right-hand corner to remove the Body Content field.  It’s not used.

  • Drag a Repeat Group object on to the Page Settings section.
  • Configure the Repeat Group with a title of Models
  • Then Drag a Node Selector control into the repeat group.
  • Configure the node selector with the title Model
  • Drag a Child Content data source onto the Data Sources area and configure it as shown below.

  • Wire/Configure the Models Node Selector control to use the Models data source by clicking on the Model control and configuring the Item Manager property in the property sheet on the right.

  • Add additional fields for X Position, Y Position, Z Position and Scale Multiplier as shown below.
    • Note the use of the input control and the configuration in the properties panel on the right.

Once you have completed this, you can click edit on the main toolbar whenever you are looking at Home scene in Preview, and you will be able to, through a simple user interface, add models to the scene.

  • Click the Crafter Logo in the upper left corner.
  • Click on Site Content in the menu at the top of the screen. This will open a panel on the left.
  • Open Pages and then click Home to load the Preview
  • Now click Edit in the toolbar at the top.

  • Click Add on Models to add your first model.
  • Now click Add on the Model element and choose “Browse for Existing.”

This will open a browser to the pokemon folder where you have been creating Pokemon content objects.  You can click Add & Close to select an item and add it to a model in the scene.

Repeat this process as many times as you need to add the items you want in your scene.

  • Note you want to alter the X Position for each model that you add so that they do not overlap when they render.

6.2 Update the Scene Controller to Use the Scene Content Model

Now that we’ve modified the Scene content model and we’ve updated our content accordingly, we need to tell our controller to look at it and populate the template variables accordingly.

  • Locate the Wrench in the upper right-hand corner on the toolbar.  Click the Wrench.  This will open the Preview Tools panel on the right-hand side of the screen.
  • Now click In-Context Editing to open the In-Context Editing tools.
  • Now click Edit Controller.
  • Now Update the code with following:

    def pokemonModelIds = contentModel.get(“models/item”)
    templateModel.pokemonInstances = []
    pokemonModelIds.each { item ->
    def pokemonInstance = [:]
    def modelItemId = item.get(“model/item/key”).text

    pokemonInstance.model = siteItemService.getSiteItem(modelItemId)
    pokemonInstance.xPosition = item.get(“xPosition”).text
    pokemonInstance.yPosition = item.get(“yPosition”).text
    pokemonInstance.zPosition = item.get(“zPosition”).text
    pokemonInstance.scaleMultiplier = item.get(“scaleMultiplier”).text

    templateModel.pokemonInstances.add(pokemonInstance)
    }

  • Click Update to save your work.

6.3 Update the Scene Template to Use the Scene Content Model

Now that our template has pokemonInstances updated to include the values updated by the author via the content model, we have to update the template to use the new values.

  • Locate the Wrench in the upper right-hand corner on the toolbar.  Click the Wrench.  This will open the Preview Tools panel on the right-hand side of the screen.
  • Now click In-Context Editing to open the In-Context Editing tools.
  • Now click Edit Template.
  • Find line 99 and make the following updatesloadModelIntoScene({
    basePath: “${texturePath?substring(0, texturePath?last_index_of(‘/’)+1)}”,
    mtlPath: “${texturePath?substring(texturePath?last_index_of(‘/’) + 1)}”,
    objPath: “${objectPath}”,
    scaleMultiplier: ${pokemonInstance.scaleMultiplier},
    xPosition: ${pokemonInstance.xPosition},
    yPosition: ${pokemonInstance.yPosition},
    xRotation: ${pokemonInstance.model.xRotation}}, scene);

  • Click Update to save your work.

Step: 7 Play Around!

Congratulations! Now anyone with access to the CMS can quickly and easily add new models and configure it into the scene.  Download additional 3D models and play around with them!

Taking It Further

Our example is very basic.  We’re simply adding user configured 3D Models on top of a live video feed. This is just the start!

Real world AR use cases typically include animation, interaction and use visual triggers like QR codes, GEO Location or pure play vision to place the augmentation into the scene.  We excluded these additional capabilities from the article because they obscure the main point of the article which is making the scene configurable by non-technical users. That said, adding triggers is very easy to do. In fact, the CMS makes triggers even more powerful by giving you the tools to understand what device the user is using, where they are (geolocation) along with other contextual information. Further, you have the full personalization capability of the CMS to help you make your AR experiences dynamic and personalized. How awesome is that?!

Conclusion

Web-based augmented reality frameworks that help developers build AR experiences are making it easier than ever for companies to leverage AR as a way to communicate, educate, entertain and assist users in a rich fashion not previously achievable without huge investments.

Today most AR work is done exclusively by technical users, and the experiences are statically designed and deployed.  This is extremely limiting.  It significantly reduces the value of the development, makes it challenging and costly to update and significantly reduces the RoI on the effort. Smart developers will separate code from content so that non-technical users can quickly and easily update the application’s content without involvement from technical resources for code updates or deployments.

Integrating your CMS with your AR is a strong pattern for solving this problem — but architects beware, not all CMS platforms are the same.  The wrong CMS will interfere with the development process and force interface and implementation details.  Crafter CMS, a modern, open source, Java-based CMS, offers the right architecture, technology, and flexibility for the job. Crafter CMS integrates with your development process and tools and puts your app in control of the interfaces and implementation code while providing content authors with the tools they need to update and publish the application independently.  To learn more check out Crafter CMS at http://craftercms.org.

Responsive vs Adaptive Web Design: Better Together

Responsive web design (RWD), the ability for your web application to “respond” to the size of the view port (traditional screen size vs mobile screen size) is common place and practice today.  Responsive design leverages client sidefront end UI frameworks to hide, show, stack, shrink and grow elements on the screen in a way that is appropriate for the size of the device’s screen.  In essence, it is many user interfaces in a single code base.

Adaptive web design (AWD), a (seemingly) alternative approach to RWD is a server side capability to respond to the client with completely unique templates / markup for a given device type (amongst other facets.)  Adaptive design, while extremely powerful is somewhat less common in part because it relies on a server and is thus a less general technology. It’s also less prevalent in part because, for a large class of problems, RWD is “enough” to get the job done.  However, as complexity and sophistication of requirements mount, adaptive design starts to show clear advantages over a purely responsive approach.

Responsive design and adaptive design are often thought of as an either/or architectural choice.  There is clear overlap in the problems they address and each approach has clear strengths and weaknesses over one-another.  Let’s take a quick look at a side by side comparison:

Comparison

Technical Approach

Responsive Design: Leverages server agnostic front end frameworks (eg. Twitter Bootstrap) to render a appropriate content flow for the given screen size.

Adaptive Design: Leverages back-end server templates (e.g. Crafter CMS) to render distinct markup for the device and context requesting the application.

Strengths

Responsive Design:

  • Mainstream approach with many frameworks that can be applied to any backend technology.
  • Single code base

Adaptive Design:

  • Truly unique markup can be delivered based on any kind of context (not just screen size.)  This means it’s much easier to deliver contextual functionality to your users.  Example:  A retail store .com site that has a traditional marketing website at mystore.com but once it’s detected that a user is inside a physical store location, it gives them a personalized in-store assistance to enhance their shopping experience.
  • Performance.  With AWD you send only the data you need for the experience you are delivering.  On slower, mobile networks that can make a big difference.

Weaknesses

Responsive Design:

  • Single code base.  The more variations you want to support, the more code and complexity there is in that single code base.  This can get out of control.
  • Performance can be an issue:  RWD sends all of the code for all variations to every consumer whether or not they will see it in their flow or not.  If the amount of code is significant, this can lead to a performance hit on slower/mobile networks.
  • It’s difficult (although not impossible) to address different contexts beyond screen size with Responsive design.  RWD was really intended to different screen sizes (view ports.)  Stretching its role beyond that takes the technology out of its wheelhouse.

Adaptive Design:

  • You need a server backend to support AWD.
  • Instead of a single UI design and template code base, you potentially have many.  If you don’t manage and factor the code properly it will be more difficult to maintain.

Opportunities

Responsive Design:

  • Provides a usable experience on any screen size.

Adaptive Design:

  • Provides a usable experience on any screen size.
  • Provide faster experiences for your users.
  • Allows you to truly contextualize the experience.  Device type/class/screen size is really just the tip of the iceberg.  Once the server is involved any type of contextual data (location, previous use, time of day, etc) can be used to deliver unique and relevant experience/features to the user.  That can be huge.

Threats

Responsive Design:

  • If requirements go beyond screen size, RWD may not be the appropriate solution.
  • If the number of variations is too high performance and code complexity may be an issue.

Adaptive Design:

  • Managing unique templates is a heavier approach that requires server technology and the skills to go along with it.

Better Together

While RWD and AWD are most commonly looked at as competitive approaches, the simple fact is that they are not.  These two approaches are in fact mutually exclusive with another and can be combined to acquire the benefits of both.

Responsive design is front end technology.   Adaptive design is backend technology.  Combine them.

Consider leveraging Adaptive design to drive truly unique and contextual markup to the user for what they are doing, on the device of their choice at that moment.   Allow the front end to leverage Responsive frameworks to make sure those experiences flow nicely on a given device.  For performance and code maintainability, as your RWD code grows in complexity and size, consider breaking it down in to smaller code bases that can be individually delivered to a device by RWD (the server.)

A simple example of this is to have AWD templates for Small, Medium and Large screens.  Each with their own unique features and markup.  When it comes to the specifics like an iPhone vs a Galaxy smart phone (both considered small), RWD steps in to make the minor adjustments required to provide a perfect fit for the given device type. This makes the code more maintainable and more importantly it makes the user’s experience much more zippy.

 

Patterns for Integrating 3rd Party Content in Your CMS

There are often times when you will want to integrate content in your site that does not ORIGINATE in your content management system.  In this article we’ll look at a few high level patterns for integration and discuss the pros and cons of each approach.

Approach 1: Sync to CMS, publish to site as full content

Approach 1 focuses the integration at the CMS level only.  Content is either pulled or pushed from the 3rd party system in to the CMS.  From there it can be further processed and is ultimately published to the website where it can be used like any other type of content including native involvement in search and targeting scenarios.

Pros

  • 3rd party content is converted to 1st class content which allows it to be used natively by the system with not rendering engine level integration.
  • Authors may augment the content coming from the 3rd party system to include additional content and metadata
  • This integration is typically very simple
  • There is no impact on the performance or complexity profile of your delivery environment

Cons

  • Syncs can be tricky.  Remember content originates in the 3rd party system but it’s possible that it may need to be further modified by the CMS.  
  • A mechanism and a process must be defined that allows content updates to flow from the 3rd party system in to the CMS.  Typically this involves a strong ID that identifies each object, a mapping of fields that can always be overridden in the CMS  and mechanism for communicating creates, updates and deletes.

Other Considerations

  • By passing through the CMS 3rd party content is not “immediately” available on the website.  Content must wait for the sync mechanism to operate. Further, there may be workflow required before content can be made live.

When to use this pattern

  • This pattern is often preferred
    • It is relatively simple
    • Allows augmentation of the content in the CMS
    • Requires no run-time integration
    • Has no impact on the performance and SLA of the run-time

When not to use this pattern

  • When any create, update or delete to the content in the 3rd party system must be IMMEDIATELY available on the live site.

Approach 2: Partial sync to CMS, full details retrieved at run-time

Approach 2 syncs a simple jacket for the content to the CMS but relies on runtime integration with the 3rd party system for the full details of the 3rd party content.  The jacket in the CMS carries only the metadata required for the 3rd party content to participate in search, targeting and other dynamic behavior on the site.   The “sync” that creates the jacket in the CMS may be done through a simple integration or can even be a manual process.

Pros

  • Less content is relied on from the CMS which means that for items that have been previously published, detailed data is always 100% in sync with the 3rd party system.
  • Search, targeting and other dynamic capabilities native to the delivery system may still be relied on for functionality within the site.

Cons

  • There is a runtime dependency on the third party system for the actual content.  If that site is down, the content is unavailable.
  • As discussed in approach 1, syncs can be tricky.  In the case of approach 2 we still have some sort of manual or automated sync to create a jacket in the CMS.
  • In this approach there are 2 points of integration 1 at the CMS and one at delivery.  When you add additional moving parts, you are adding operation complexity that must be maintained and supported.
  • Content can be out of sync between what the site has in it’s jacket (and this in the indexes) and what is in the 3rd party system.  You must consider all cases: create, update, and delete

Other Considerations

  • By creating a run-time dependency between the delivery system and the 3rd party system you dictate that the 3rd party system must (generally) have the same reliability and performance capabilities of the delivery system.

When to use this pattern

  • When detailed aspects of the 3rd party content must be IMMEDIATELY up to date.
  • When it is impractical to import all of the 3rd party content in to the CMS.  A typical use case for this approach is 3rd party hosted video.  The CMS may contain only basic metadata and thumbnails while the 3rd party system retains all of the various video encodings.

When not to use this pattern

  • If the website and the 3rd party system must be 100% in sync on ALL aspects of the content at all times
  • When approach 1 is viable.

Approach 3: Run-time integration only

Approach 3 focuses integration at the content delivery tier only.  Display and behavior related to the 3rd party content is all handled through integration.

Pros

  • Approach is the most simple from a “moving parts” perspective. There is no syncing involved
  • 3rd party content is immediately up to date when changes are made in the 3rd party system

Cons

  • There is a runtime dependency on the third party system for the actual content.  If that site is down, the content is unavailable.
  • All content display, and behavior such as query driven listings, content targeting, and search must be integrated.
  • It can be difficult to integrate functionality like search where CMS content and 3rd party content are placed together on the page in a seamless way.  This is because the implementation is very likely to require issuing at least two queries (one to the CMS and one to the 3rd party system.)

Other Considerations

  • By creating a run-time dependency between the delivery system and the 3rd party system you dictate that the 3rd party system must (generally) have the same reliability and performance capabilities of the delivery system.

When to use this pattern

  • When 3rd party content must be IMMEDIATELY available and up to date on the site the instant it is updated in the 3rd party system.
  • When there is no need to rely on native capabilities of your delivery platform for delivery of the content and dynamic behavior (like search.)

When not to use this pattern

  • When option 1 or 2 are viable and use of native capabilities of the content delivery platform useful / required.

 

Querying Content in Crafter CMS

Types of Content Queries

Crafter CMS supports 3 specific types of content queries:

  • Cross content Lucene/Solr queries. This enables you to query any/all content objects, by any group of properties.)
  • Filtered Structural Queries. This enables you to query against the repository structure e.g. “Get all articles by author XYZ”
  • Content Item specific query. This enables you to write queries inside of a given content item.

Type 1: Make a Lucene/Solr Query

The following code examples use the Crafter Search Service in Crafter Egnine to get content. You can find the interface for this service HERE.

def queryStatement = 'content-type:"/component/article" AND author:"Russ Danner"'

def query = searchService.createQuery()
query = query.setQuery(queryStatement)

def executedQuery = searchService.search(query)
def itemsFound = executedQuery.response.numFound
def items = executedQuery.response.documents

return items

Type 2: Make a Query for Content Based on Structure

The following code examples use the Site Item Service in Crafter Egnine to get content. You can find the interface for this service HERE.

def topNavItems = [:]
def siteDir = siteItemService.getSiteTree("/site/website", 2)

if(siteDir) {
    def dirs = siteDir.childItems
    dirs.each { dir ->
            def dirName = dir.getStoreName()
            def dirItem = siteItemService.getSiteItem("/site/website/${dirName}/index.xml")
            if (dirItem != null) {
                def dirDisplayName = dirItem.queryValue('internal-name')
                   topNavItems.put(dirName, dirDisplayName)
            }
   }
}

return topNavItems

Make a Query for Content Based on Structure with Filter

The following code examples use the Site Item Service in Crafter Engine to get content. In the example we build on the Site Item Service of getting objects under a specific tree in the repository by supplying a filter that will be applied to each object first to determine if it should be part of the result. Filters can make their determination based on the path or the content or even “outside” influence.

  • You can find the interface for this service HERE.
  • Note in the example below we define our own filter based on the ItemFilter interface found HERE.
  • However, you may use out of the box filters as well if they meet your needs. These are found HERE.
  • Finally be aware that for simple filename patterns, methods for this already exist in the Site Item Service and no filter is required (but they make for an simple to understand example.)
import org.craftercms.core.service.ItemFilter
import org.craftercms.core.service.Item
import java.util.List


def result = [:]
def navItems = [:]
def siteDir = siteItemService.getSiteTree("/site/website", 2, new StartsWithAItemFilter(), null)

if(siteDir) {
    def dirs = siteDir.childItems
    dirs.each { dir ->
            def dirName = dir.getStoreName()
            def dirItem = siteItemService.getSiteItem("/site/website/${dirName}/index.xml")
            if (dirItem != null) {
                def dirDisplayName = dirItem.queryValue('internal-name')
                   navItems.put(dirName, dirDisplayName)
            }
   }
}
result.navItems = navItems

return result


/**
 * Define a filter that returns only items that have a name that starts with "A" or "a"
 */
class StartsWithAItemFilter implements ItemFilter {

    public boolean runBeforeProcessing() {
        return true
    }

    public boolean runAfterProcessing() {
        return false
    }

    public boolean accepts(Item item, List acceptedItems, List rejectedItems, boolean runBeforeProcessing) {

      if (item.getName().toLowerCase().startsWith("a")) {
          return true
      }

      return false
    }
 }

Type 3: Make a Query Against Fields in a Content Object

The following code examples use the Site Item Service in Crafter Engine to get content. You can find the interface for this service HERE.

def result = [:]
def segment = "a segment value" // could come from profile, query param etc

// load a specific content object
def itemDom = siteItemService.getSiteItem("/site/components/sliders/default.xml")

// query specific values from the object
result.header = itemDom.queryValue("/component/targetedSlide//segment[contains(.,'" +  segment + "')]../label")
result.image = itemDom.queryValue("/component/targetedSlide//segment[contains(.,'" +  segment + "')]/../image")

return result