Working with Content as XML/DOM in Crafter CMS

In a previous article (Querying Content in Crafter CMS) we talked about how you can query content in Crafter using content and search services. Under the hood, Crafter CMS saves all of the content you create through the authoring interface (Crafter Studio) as XML.  This XML is published from Crafter Studio to dynamic delivery engines (Crafter Engine) and is available through the content services and is also indexed in Solr and thus available through Crafter’s search services.  There are times in the rendering engine (Crafter Engine) when you may want to get access to values in the XML directly and work with the content as an XML Document Object Model (DOM) API.

In the example below I will show you how you can:

  1. Load a content object
  2. Get access to its DOM
  3. Query for DOM elements within the parent object
  4. Process the results

To illustrate this let’s imagine that we have a use case where content authors need to manage values like CEO name or Number of Offices throughout a site that might change.  They want to pepper these values throughout the site but want to update them quickly from a single place.    To do this we would use a macro/placeholder type approach in the content where the author would enter content with macro/placeholder in the place where the value should go.

Example:

Acme Anvil Co has [Office_count] offices throughout the world.

To manage the Macro/Value pairs we’ll give the authors a content type:

Once you have your macro type defined you can create a content object based on that type:

Now that we have created an author managed list of macros and their corresponding values.  Let’s create a basic controller (some assumptions to keep things simple) to process the content in a component and replace the macros found in the content with the value.

Controller example:

// Load the content object containing the macros key values
def macroItem = siteItemService.getSiteItem("/site/components/5ef8d326-3e85-9f67-e8d6-47fb5dfba21d.xml")

// Get the unprocessed content
// Here we'll put some assumptions in the code.  In reality you may want 
// to process all the content fields in the content.
def content = contentModel.queryValue("content_html")
def contentUpdate = content
// Query the macros and iterate over them performing the replace. 
// this is where we get the DOM and leverage the DOM API
def macros = macroItem.getDom().selectNodes("//item")
macros.each { pair ->
 def macro = pair.valueOf("./macro")
 def value = pair.valueOf("./value")
 contentUpdate = contentUpdate.replace("["+macro+"]", value)
}

// Make the processed content available to the template
templateModel.content_html = contentUpdate

 

You can easily tie this controller into any template by adding it to the top of the template:

<@controller path="/scripts/controllers/macro-controller.groovy" />

Most of the time with Crafter CMS, there is no reason to access the DOM directly.  From time to time you may find it more convenient or necessary.  Now you know how!

Working with Your First Crafter CMS Website

This article assumes that you have followed the steps in the Cafter CMS Quick Start Guide to get Crafter CMS installed and logged in to Crafter Studio. We will be using an out-of-the-box blueprint, called “Website_Editorial” to create your first website.

Let’s get started building your first website!

Creating your website from out of the box blueprint Website_Editorial

After logging in, you’ll see the MySites screen (Below). Click on Create Site

Your First Website - Sites ScreenGive the site a friendly name for the Site Id , a description and then choose a blueprint. We’re going to be using the “Website_Editorial” blueprint. Blueprints offer you a starting point for your website. New blueprints can be created and installed into the system. As you are entering the site id, spaces are removed and upper case letters are converted to lower case letters.

Your First Website - Create a SiteClick on Create and wait for the system to create your site based on the blueprint. It’s creating the following: configuration, site content, and permissions based on the template provided by the blueprint.

Your First Website - Creating a Site Spinner DialogWhen it’s done you will be taken to the Home Page of your site: (need to update image below)

Your First Website - Home PageYour site is setup, we can now start adding/editing content! To edit content you see on the page, click on Edit at the top (see above). This will open a form (see below) where you can edit the page content. To see other ways of editing page content, see Editing a Page.

Your First Website - Editing Content

Adding a new article page to the site

We’ll be adding a new article to the site. To add a new article (or a new page), navigate to the level and location within the site navigation tree in the Sidebar where we want to create the new page. In this case, we are adding an article under articles -> 2017 -> 3. Right click, then select New Content (need to update image below)

Your First Website - New ContentWe’ll then select the page template we want. Since we are adding a new article to the site, we will be selecting the template Page – Article

Your First Website - Select Page TemplateWe’ll start filling out the form for our new article, “Where to find cherry blossoms in Virginia”. For the Page URL, replace spaces with dashes. You can write the Internal Name and Title however you like as long as it is 50 characters or less as indicated on the right of the input boxes. For the Headerand Sidebar, we will be using the default provided by the template.

Your First Website - Page PropertiesThe next section on the form is the Metadata section, where we can select the category for our article, the targeted segments of the article and whether our new article should be added to the Featured section. Our new article, will be under Entertainment for the Categories and the targeted segments is Gal. We will also be placing our new article in the Featured section.

Your First Website - Page Metadata SectionFinally, we add our blurb in the Content section of the form. Here, we fill out the SubjectAuthorDateSummaryImage and Section, which contains the content of our article.

Your First Website - Page Content SectionHere’s the site, with our newly created article in the featured section.

Your First Website - Newly Created Site Home PageYou can add more pages or modify/remove the existing pages from the blueprint, depending on your needs. To remove or edit an existing page, navigate to the location of the article you want to edit/remove. Right click on it, then select the action you would like to do on the page.

Your First Website - Edit a Page

Updating the Contact Us section in the sidebar

Another thing that we may want to modify from the blueprint, is the left rail. For this example, we are going to modify the Contact Us section in the left rail (contact widget). To edit items in the sidebar, click on the pencil on the top right of your screen to enable in-context editing. Pencils should appear on sections editable on the page. Go to the top left of the left rail and click on the pencil there.

Your First Website - Edit the Left RailA form with all the editable content of the left rail will appear. Go to the Widgets section of the form, select Contact Widget and then click on the Edit button on the right of the list of widgets.

Your First Website - Left Rail FormA form containing all the editable fields in the Contact Us section will appear. Modify the fields that you want to change.

Your First Website - Contact WidgetHere’s the sidebar with the Contact Us section updated.

Your First Website - Updated Contact Us Section

Editing the features section, “Erat lacinia”

We will now edit the features section in our blueprint. The features in this section has been configured as components, as you will see in the images below. There are multiple ways of editing the features section in the blueprint.

Your First Website - Add Features through Drag and DropWe’ll start out by adding a feature using the pencil at the top of the features section (In the image above, we will use the pencil captioned “Edit the whole features section”). Click on the pencil at the top of the features section. A form will open containing the content of the section. As you can see in the image below, there are currently two features in the section.

Your First Website - Edit by Clicking on the PencilIn this form, you can add another feature, by clicking on the Add button, which will give you a menu to Create New – Features or Browse for Existing – features

We will add a feature by selecting Create New – Features as seen on the image above. This will open a form, where we will now enter our content.

Your First Website - New FeatureYour First Website - New Feature AddedWe will now add another feature, by selecting Browse for Existing – Features, after clicking on the Add button. This will bring up a form containing a list of existing features in the blueprint. Select one, then click on Add & Close or, click on the radio button of your selection, then click on Add Selection. This will add your selected existing feature to the features section of the page.

Your First Website - Browse for Existing Features ComponentWe will again add another feature, this time by opening the Preview Tools panel, and then clicking on Page Components. A Components panel will open where the Preview Tools panel used to be, containing components that you can drag and drop onto the drop zone highlighted on the page. To click and drag a new feature onto the drop zone, click and drag Feature, under the general heading. This will then open up a form for you to add your new feature content. To click and drag an existing feature onto the drop zone, click on Browse Features. This will then open up a form containing a list of existing features that you may choose from. Make your selection, the form with the list will then close and now you can drag and drop your selected existing feature onto the drop zone.

Your First Website - Drag and Drop ZoneFrom inside the drop zone, you may also re-arrange the features by clicking and dragging on a feature and placing it in your desired position. Notice the positioning of the newly added feature and the existing features, which have been re-arranged compared to the previous image. To delete/remove a feature from the drop zone, just click on the X as show in the image below.

Your First Website - Drag and DropTo edit a feature, select a feature from the list and click on the Edit button on the right of the list. Edit the fields you would like to modify, then click on Save and Close or Save Draft to save your changes or click on Cancel to discard all changes in the form. You can also edit a feature by clicking on the pencil next to the feature when In-Context Editing is enabled.

Your First Website - Edit FeatureTo remove a feature, select a feature from the list and click on the X button on the right of the list. Or, from the drag and drop zone when you click on “Page Components” in the Preview Tools panel, click on the X next to the feature.

Your First Website - Remove Feature

Publishing Your New/Edited Page

Your site is published after creating the site from the Website_Editorial blueprint. If you make edits to any of the pages or created new pages, it will need to be published for your site visitors to see the changes. There are a couple of ways to publish your page edits. The first thing you need to do is to navigate to the page you want to publish in the Site Navigation Tree (Enabled by clicking on Sidebar on the right of the Crafter CMS logo on the upper left hand corner of Studio). After navigating to the page you want to publish, there are two ways to publish:

  • Click on the page you want to publish. In the context menu, click on Approve & Publish
  • Right click on the page you want to publish from the Site Navigation Tree, then click on Approve & Publish

Your First Website - Publish Your New ContentYou will then prompted whether you want to publish the page now (Items should go live now), or publish the page at a later date and time (Items go live on a specific date & time).

Your First Website - Publish OptionsFor more information on content authoring, please see the documentation section: Content Authoring

Great Crafter CMS Webinars

Check out these free pre-recorded online events! Learn more about how Crafter CMS is built and where and how to apply it to your projects!

 

Planning a Smart Mobile App Development Strategy

Building and Running Crafter CMS from Source. It’s Simple!

Crafter CMS is an open source content management system for Web sites, mobile apps, VR and more. You can learn more about Crafter here.  In this article you will learn how to build Crafter CMS from source as well as how to start and stop the services.  It’s very easy!  Thanks to Gradle and a very easy install process, you will be up and running in 3 simple command line operations!

0. Prerequisites

You must have these prerequisites on your system before you begin:

  • Java 8
  • Git 2.x+
  • Maven 3.3.x+

OS X extra prerequisite

  • If you’re on OS X, then using brew install the latest openssl formula, like this: brew install openssl

1. Clone the Crafter repository

git clone https://github.com/craftercms/craftercms.git

This will create a directory called craftercms.  Change path in to this directory.

2. Build the Environment

Crafter CMS is built along a microservices architecture, and as such, comprises a number of head-less, RESTful, modules that work together to provide the final solution. In this section, we’ll start with the simple case of build everything/run everything, and then move on to building/hacking individual modules.

Build all Crafter CMS modules:

    ./gradlew init build deploy

3. Start and Stop All Services

Start Crafter CMS:

   ./gradlew start

You can now point your browser to http://localhost:8080/studio and start using Crafter CMS. To get started with your first Crafter CMS experience, you can follow this guide: http://docs.craftercms.org/en/latest/content-authors/index.html.

Note
  • The authoring environment runs on port 8080, a great place to start, while the delivery environment runs on port9080.

Stop Crafter CMS:

    ./gradlew stop

Two Environments: Authoring vs Delivery

You might have noticed that you essentially have two environments built and running: authoring and delivery. Crafter CMS is a decoupled CMS, and that means you have an authoring environment that caters to content creators, and a different environment, delivery, that handles the end-users that use the experience created by the former.

As a developer, you can use an authoring environment for most tasks without the need to run a delivery environment. It’s important to note that delivery essentially runs the same software that’s in authoring except Crafter Studio (the authoring tools). By default, this project will build both environments unless instructed otherwise.

The authoring environment runs at http://localhost:8080/studio, whereas

the delivery environment runs at http://localhost:9080/

Start, and Stop a Specific Environment

To start and stop one of the two environments is similar to building/starting/stopping All.

Authoring

    ./gradlew start -Penv=authoring
    ./gradlew stop -Penv=authoring

Delivery

    ./gradlew start -Penv=delivery
    ./gradlew stop -Penv=delivery

Ready to Rock

In just a few command lines and a few minutes you are now ready to rock!  Use Crafter Studio to create new projects or alter and build the source code to make changes to the platform itself!  Happy crafting!

Creating and Installing Project Blueprints in Crafter CMS

What are Blueprints?

Blueprints are Crafter CMS project templates. It provides an initial structure/layout for your site containing one or more of the following: content types such as pages and components as described in Content Modeling, static assets such as images, videos, etc., and site configuration files for managing items in the blueprint such as taxonomies (categories, segments), roles, permissions, etc.

Cook Books - Blueprint AnatomyIn the blueprint that comes out of the box with Crafter CMS, Website_Editorial blueprint, it provides us with an initial structure for our site, along with the site navigation, content inheritance, taxonomies for organizing the content such as categories and segments, which was also used for targeting content, static assets such as the initial images and fonts used for the site and configuration files for managing things like the personas for targeting, the permissions for all the items in the site, the role mappings, the RTE configuration, etc. To see more of the Website Editorial blueprint, please see Your First Website where we create a site based on the Website_Editorial blueprint.

As mentioned earlier, blueprints allows us to generate sites with predefined layouts, contents and configuration. Blueprints could be a site theme or an API only site. New blueprints can be created from a site and added into Crafter CMS allowing the creation of more sites based on the new blueprint. In the section that follows, we will see how the Empty blueprint that comes out of the box from Crafter CMS and an existing site is used to create a new blueprint.

How do I make my own Blueprint?

Start by Installing and Starting Crafter CMS.

Blueprints are almost the same as a site. So, you can use a new site created from the Empty blueprint as the starting point for your blueprint. (See Your First Website but create it from the Empty blueprint).

Adapting an HTML template

If you have an existing pure HTML template (and if you don’t, you can find free ones, even with commercial friendly licenses like MIT and some flavors of Creative Commons), you can adapt it into a blueprint.

Cook Books - Template AnatomyGenerally, pure HTML templates have a file structure similar to the picture above. To start, you’ll want to copy all files except for index.html and any other .html files to your site’s static-assetslike this:

Copy folders to static-assetsHTML files will become Freemarker templates. For this cookbook, you’ll see how to adapt an index.html page, then you’ll be able to adapt other pages. Start by editing the main page’s ftl template, and replacing its contents with the index.html’s contents:

Copy index.html contents to page ftl file.You should keep <#import "/templates/system/common/cstudio-support.ftl" as studio /> at the very start, and add <@studio.toolSupport/> right before the body tag closes to have proper Studio support. Next, all resource locations are probably pointing to the wrong location. To fix this, replace every relative url that doesn’t point to a page (this would include <link rel="stylesheet" href="tags for CSS files, <script src=" for JS files, <img src=" for image files, and <source src=" for video and sound files) such that it starts with /static-assets/ and points to the corresponding file.

Modify the Rich Text Editor configuration so it uses your template’s stylesheets. See Rich Text Editor (RTE) Setup

At this point, you should have a static page that looks just how the template is supposed to look. For every other HTML page, you have to either create a new page content type and, like with index, replace its ftl template with the page’s source; or, generalize the content type, with proper content modeling, such that multiple pages share a single ftl template and vary only in the components they contain. Let’s see some tips for this.

Content Modeling

A powerful and extensible blueprint that can be used in a variety of pages and scenarios needs proper Content Modeling, so you have to be familiar with it before proceeding.

A good blueprint separates each meaningful chunk of HTML code into a component. For example, whether you implement an “Our Team” section using a repeating group or multiple “Teammate” child components, it still has to be a separate type that only contains information related to “Our Team”. Whether it is a Component or a Page, it shouldn’t contain “Product” information. Once you have identified HTML chunks with a meaning, start by moving them into their type’s template.ftl. Next, replace any information with a variable from the contentModel (and add the respective control to the Content Type). Unless they are extremely simple, most pages will contain child components, even if they are just a header and footer component provided by the Section Defaults.

There are some best practices to help you:

  • Prefix all your Content Type’s display label with either “Component – ” or “Page – ” as appropriate.

  • Make use of Section Defaults. Most sites will have a site logo that will be used all throughout the site, this is a perfect use case for Section Defaults.

    • Additionally, since Section Defaults have inheritance mechanics, a child folder that’s meant to have private pages could have it’s own Section Defaults that overrides the normal site logo with a more private looking one, signalling users that they are in the intranet.
    • You can apply this similarly for headers, footers, log in floating forms, and many more.
  • Use drag and drop but keep it to a minimum. At the moment, you can’t limit what kind of components can be dropped into a container, so this enormous amount of flexibility can make for a confusing user experience. Picture having a page with a group of sections, that each contains headers. If both sections and headers are drag and droppable, an user could accidentally drop a section inside another section without noticing instead of just reordering. It could be more comfortable that only sections are drag and droppable.

  • You can use label controls to add additional information to the content type’s form. This is useful to add tips or additional information for advanced controls.

  • Prefer repeating groups over child components. Child components are ultimately more versatile, but if you are only going to repeat text, and that text is not going to appear outside the repeating group again, it’s a better user experience to just use a repeating group.

    • Bear in mind that you can’t have nested repeating groups, so only the innermost repetition can be a repeating group.
  • You can set up folders for specific content types, and you can enforce them by using <paths> in your types’ config.xml. Use includes whenever you want to whitelist some paths, and use excludes to blacklist some paths, but do not mix them. For more examples, see Content Creation Permissions

    <paths>
        <includes> <pattern>REG_EXP_HERE</pattern> </includes>
        OR
        <excludes> <pattern>REG_EXP_HERE</pattern> </excludes>
    </paths>
    
    • You can also use this to enforce single page blueprints by using <excludes> <pattern>^/.*</pattern> </excludes> in your page type’s config.xml, effectively forbidding from creating a new page.
  • Ensure your blueprint supports In-Context Editing.

  • For most sites, you’ll have to override Studio’s default navigation menu tags. You can do this by reading Rendering navigation.

Above all, blueprints should be usable and simple.

Packaging

Suppose {CRAFTER_HOME} is the path to your Crafter installation so that it contains the startup scripts, apache-tomcat/ and data/ folders.

Blueprints reside in {CRAFTER_HOME}/data/repos/global/blueprints since Crafter 3.0. Each folder corresponds to a blueprint (You may notice the empty and website_editorial blueprint folders), you can start by copying the empty folder and renaming it to your blueprint’s name, like “my_blueprint”.

Your site exists in {CRAFTER_HOME}/data/repos/sites/your-site-name. Inside, you’ll notice 2 repos, sandbox and published. Inside of either of them, lie the site’s folders, but since sandbox contains your site as it currently exists in your Studio preview, we’ll be grabbing the files from this one. You need to move this site’s folders into an external folder named as your blueprint, but avoid copying the .git/ folder contained there, as it’s unnecessary for the final distributable package and may even contain sensitive information.

Note

Don’t merge folders, before copying any folder, delete the existing one so any renamed or deleted files don’t persist.

Copy ``scripts/``, ``site/``, ``static-assets/``, ``templates/``In the previous screenshot, we didn’t copy the config/ folder. Why? (Warnings). You can either:

  • Copy the config folder and modify permission-mappings-config.xml and site-config.xml to use {siteName} again as explained in (Warnings)
  • Keep config as is and only copy the files you’ve modified. This will likely include the whole config/studio/content-types/ folder and config/studio/preview-tools/components-config.xml for drag and drop.
  • Keep your blueprint in a VCS which will allow you to compare it against your changes and interactively see when to preserve the old version. This will also help you make any updates when blueprints get updated. You can either use git or a visual diff tool.

Now that you have merged your “site” with the Empty blueprint in the proper way, the resulting folder is ready to be distributed. To install, follow the next steps.

Installing

On Crafter 3.0

  1. Copy your blueprint folder into {CRAFTER_HOME}/data/repos/global/blueprints.

  2. Once you do, commit the change to the global repo ({CRAFTER_HOME}/data/repos/global/) by using git, and your blueprint will now start appearing when you try to create a new site.

    • Crafter 3 uses a vanilla version of Git, so regular git commands work as intended. To commit your changes so Crafter can see it, head to {CRAFTER_HOME}/data/repos/global/blueprints and git add your modified files like this

      git add <filename>
      

      for each filename. Or, to add all at once use:

      git add --all
      
    • And once you are done, commit them with the following command:

      git commit -m "<the commit’s description>"
    • No need to push, there’s no remote configured. You can also use any git client. Now, it will be available when you create a new site.

Using the Blueprint

Once you’ve installed your blueprint using it is simple!

After logging in, you’ll see the MySites screen (Below). Click on Create Site

Your First Website - Sites Screen

Give the site a friendly name for the Site Id , a description and then choose a blueprint. We’re going to be using the “Website_Editorial” blueprint. Blueprints offer you a starting point for your website. New blueprints can be created and installed in to the system. As you are entering the site id, spaces are removed and upper case letters are converted to lower case letters.

Your First Website - Create a Site

Click on Create and wait for the system to create your site based on the blueprint. It’s creating the following: configuration, site content, and permissions based on the template provided by the blueprint.

Your First Website - Creating a Site Spinner Dialog

When it’s done you will be taken to the Home Page of your site:

Your First Website - Home Page

Reindexing Content for Search and Queries in Crafter CMS

It is necessary from time to time to reindex content due to schema changes, migrations, and other scenarios. A bulk deployment will push all content to your index but involves several steps in addition to indexing which may not be needed. This article shows how to use the deployer to (re)index content that has already been deployed.

Reindexing the site content can be done using the reprocess feature in Crafter Deployer.

Step 1: Delete any existing content in the index

curl "http://hostname:port/solr-crafter/update/?commit=true" -H "Content-Type: text/xml" -d "<delete><query>crafterSite:MYSITENAME</query></delete>"

Step 2: Invoke the reprocessing

curl http://hostname:port/reprocess?password=MYPASSWORD&target=MYTARGET&processor=MyBeanName
Parameter Name Description Example
hostname Deployer’s hostname localhost
port Deployer’s port. 9191 default is 9191
password Deployer password
target The name property value of a target bean in target context file. demodotcomprod
processor The bean name of a processor. For reindexing, it should be the bean name of a search update post processor registered for a target bean in target context file. (e.g. SearchUpdateFlattenXmlProcessor) DemoDotComProdSearchProcessor

Step 3: Wait for indexing

You will see indexing activity in the deployment log as well as entries on the server(s) running Crafter Search and Solr. Indexing activity time is dependant on the amount of content which must be re-processed.

Reindexing Content Without Disrupting Service in Production

In some scenarios, it’s not possible/appropriate to delete a live index and wait for the index to rebuild in production. Perhaps the index is driving dynamic features on the site that will break while the index is empty or being rebuilt. In these scenarios, you need a process for building the index offline and swapping it in.

Step 1: Prepare a re-index core

The first step is to prepare an additional empty core on Solr where you can index the content.

Step 2: Set up a new deployment context that points to the new core

The next step is to create a new deployment context that mimics/is a copy of the production deployment context but that points to the “re-index” core. Note that you must restart the deployer in order for the new context to be detected.

Step 3: Content freeze

Once you are about to start a re-index you need to freeze your authoring/editing activity. If Content is being updated in the live environment while you are rebuilding your indexes, you may miss updates. Ask your authors not to publish during your re-index process.

Step 4: Re-index

Following the process above “Reindexing Content for Search and Queries” for re-indexing content, you want to invoke a reprocess action against your new deployment context.

Step 5: Wait

You will see indexing activity in the deployment log as well as entries on the server(s) running Crafter Search and Solr. Indexing activity time is dependant on the amount of content which must be re-processed.

Step 6: Swap indexes

Now that indexing is complete you need to load the re-indexed content. Follow these steps * In the solr console for the core administration click swap cores and provide the paths to the new index. * Once the core has reloaded, move the original core to backup * Consider creating a copy of the re-indexed core with the original name and swapping again to preserve file/folder names.

Step 7: Unfreeze Content

Now that you are certain everything is working as it should, notify your authors that they may start editing and publishing activities.

Step 8: Clean up

Now that your process is complete you can clean up some of the artifacts created by the process. * The re-index core if swapped out * The new deployment context