The Shortcode to HTML Town Hall

[charlie]

Hello all,

We’re excited to share more about a project we’ve been eyeing for sometime — shifting Cornerstone from a shortcode output to that of HTML. There are several reasons behind this, which we’ll share below, including a specific example with how it will improve our integration with 3rd party plugins.

Over the years, we’ve experienced a number of issues related to how other plugins read our content. Pretty much from the beginning, we have stored our content documents as shortcodes (posts and pages, everything but layouts) . The reasoning behind this was so that other plugins would be able to interact with our content. This was also the “standard” way of working with WordPress several years back. Unfortunately, there are a number of issues with this approach in the modern area. For starters, plugins these days simply don’t read shortcode content. Furthermore, when we have tested rendering shortcodes in various SEO plugins we have encountered numerous performance and content related issues. It is also not a good use of time for us to go through individual plugins and create custom integrations.

The Better Way

Say hello to the HTML Storage System, a change in Pro 6.6, and Cornerstone 7.6 products. This is how modern builders store their content and, from our internal testing, this change fixes a number of issues we have outlined above in addition to allowing the following plugins to be able to read our content when in place. This is not a complete list and a larger list will be released as we get through beta testing.

• Rankmath SEO
• All in One SEO
• Jetpack Search
• WordPress Searching by post content

The Upgrade Path

Upon upgrading a Pro or Cornerstone site to this and subsequent versions, you will still be in shortcode mode. For any new sites, we plan on setting HTML storage as the default. We are doing this to ease the upgrade process and work out any potential issues with more legacy sites. This is the prefect time to introduce this as we prepare for the dashboard upgrade coming in Pro 6.6 and Cornerstone 7.6 products. You will have an option to control this storage system for older sites with the migration running in the background or through a button. If you are having an issue with a plugin that this system fixes, we will be telling you to set up a staging site and then turning this on to troubleshoot.

Faster Pages

In addition to being a more modern architecture, storage in HTML also gives us the opportunity to speed up your pages. We plan on adding an additional setting to pages which will let you output cached HTML content. The reason for not always outputting a cache is that certain Dynamic Content and Loopers can change outside of the page content. If you have simple pages, this feature will act as a way to speed up your pages natively in Cornerstone through bypassing our Element rendering system and relying on pre-built HTML, giving you similar speeds to that of a caching plugin but without the need of a caching plugin. We take our position as the most advanced builder on the market today very seriously, and this is a big example of why using a builder that is supported and improved can make a significant difference in both the speed and efficiency of building websites. Now you will be able to output direct HTML while still leveraging the power and extensibility of Cornerstone.

Example: The Future of WPML

To further explain this new direction, let’s take a look at one major pain point over the years: the WPML integration. Unfortunately It is constantly at odds both with itself and what we are trying to accomplish through the integration. A lot of our users end up recommending Polylang as a result, a plugin which we have written zero integration code for. If a plugin with no integration code is out performing something we have written a custom integration code for, then we are clearly on the wrong path. Part of this is due to the architecture of WPML and why we are deciding to make a number of changes to this system.

Currently WPML reads our raw Element JSON and converts content based on that data. This has been nothing but problematic as it means every new Element or change needs to also go through WPML so they can update their integration which takes time and resources. Global Blocks / Components have never been able to auto translate and at this point it’s safe to say WPML will never support Global blocks for this feature.

We are also planning on changing how the language switcher works and how WPML works with Cornerstone. This is a feature we believe at this point was done improperly. In the current system, any change to a layout or content that is design-based, must be redone in the translations. This ends up putting us in a position where we are the translation system and not the page builder.

This has caused us to spend hundreds of hours trying to get the two to work together in a non-standard way and it isn’t working for anybody. We plan on changing the language switcher in the future to either be removed or repurposed to send you to the WPML backend for your particular post. If you really want a different design per language (which we never see), we will add in additional localization dynamic content and conditions, further building on our page building power. The HTML storage system adds the possibility of a new paradigm for the future or Cornerstone, especially as we prepare for the introduction of CSAI and other important features. Now translations will be handled by the plugin actually tasked with doing the translations. WPML can work the same way it does with the block editor or Elementor, reading HTML and parsing content in HTML then giving you a way to translate yourself or through automatic translations.

This also leads to other exciting opportunities. If we start storing content for Layouts and Components with HTML (which don’t store them as either shortcodes or HTML currently) we can finally solve the issue with auto translating headers, footer, and Components.

We do not think these WPML changes will be completed in Cornerstone 7.6, but after we introduce the HTML storage system, we plan on working towards this goal of a streamlined and standardized WPML integration that everyone can enjoy. We are currently still in the brainstorming phase of what this could look like and would love to get your thoughts.

We hope you are as excited these new improvements and are happy to answer any questions or comments. We are planning on starting beta testing at the end of the year and releasing early next year. As always, we look forward to keeping you on the cutting edge of website building, both now and in the future.

[devinelston]

Very cool!

One question that immediately occurs to me is how would headers that contain dynamic content work? For example, a small business site that dynamically displays the current day’s opening hours, etc… Similar to the “cached output” for pages, would there be an explicit option for headers as well?

[henrybag]

As someone who is about to use WPML for a new site and the other people have abandoned it for another plug-in would you suggest persevering as future updates will make it more compatible? It’s not a project I want too much stress with if you know what I mean.

[charlie]

Layouts are special place where they don’t store shortcodes and their Post content is actually the Cornerstone Element JSON. We can certainly take a look at caching layouts like Headers and Footers too. For the reason you brought up about Dynamic changes in content is the reason no content will be cached by default, as to prevent any issues from happening. Much like the External REST API we will probably have a selector for how long to keep a cache as well.

If you keep in mind Components in WPML can’t be translated properly without using Twig’s __ function and the string translation plugin you can get your work done. Most of our short term goals with WPML is to fix some of the editor issues within Cornerstone. Such as you need to use the Preview Manager in certain cases to get a proper Cornerstone Preview. I layout what mostly needs to be done with Components as well here.

[raye]

Hey @charlie, just a little feedback from my side.

I think it’s about a year ago that I decided to write an E-Mail to you all at Themeco where I pointed out my concerns about WPML. At that time, I was in the middle of a bigger international website project and I needed to change to Polylang midair because WPML deleted constantly some Yoast MetaDescriptions upon saving and did some other unreliable things – it was no joy.
I was really concerned since I’m using Pro for all of my web projects – and I love it. But the lack of WPML ment no multi language projects for me, or going the unofficial Polylang way.

So I’m very happy to hear that you are aware of this problem and that you are going to tackle it! Although I don’t like WPML – I think it is way too much overkill for many projects and really would like to use a lighter system. But it is a good sign in terms of product care. This gives me, as a user of your product, a good feeling about it.

Not to mention all the other great latest additions, like Super Loopers and TWIG. In fact, your development cycles feels very active to me. Maybe even too active for my personal pace – it’s getting hard to learn every new feature in depth
I’m happy you constantly keep things getting better. I am also very curious about the HTML switch and if there will be noticeable performance gains.

So thank you and to the whole Themeco team for all the work you put into this project. It allows me to make a living from the things I love and from the tools I’m convinced are good solutions.

Cheers, René

[deranaloge]

hi charlie,
that are great news! Thanks for the constant development of pro!

@charlie

If you really want a different design per language (which we never see), we will add in additional localization dynamic content and conditions, further building on our page building power.

I use that one some sites, so there must be a way to use this on new AND sites where we upgrade pro.
Please have this in mind.

The move to HTML is great and hopefully all problems get solved
i use: https://theseoframework.com/ for most of my pages because it is “lighter”. Will this be also “supported”?

thanks again

[JvP]

Great move I think. Potentially better performance, better data portability since the output will be HTML and easier 3rd party integrations. I’ll be testing the SmartCrawl SEO plugin with this when the time comes.

[charlie]

Glad everyone is excited for the changes. I’ll think keep in mind the points you all have brought up about WPML.

The SEO Framework and Smart Crawl worked great in HTML mode. I’m happy to check any plugins you want me too before the beta testing.

[quentosity]

Any testing being done with Caching plugins like WP Rocket? I assume shouldn’t be any issues but just thought mention as I see you mention about it outputting it’s own cache so wonder if certain setting is best to work with other cache performance plugins.

[charlie]

Caching plugins usually bypass the page creation process entirely so I’m not too worried about it. I can run some tests though.

I’ve played around with caching Cornerstone pages a little more and the speed results weren’t that impressive. About .04 seconds gained at most. More advanced pages might benefit a little more and I would still use a dedicated caching plugin that would also cache other plugins. I might focus our efforts on looking at other performance bottlenecks in Cornerstone. Getting others plugins to work properly is a bigger goal of this project.

[deranaloge]

hi there,
any eta for 6.6?
thanks in advance
harald

[spedney]

Hi @charlie I was thinking about an issue with SEO plug-ins and cornerstone. I noticed last week that for rankmath the xml sitemap does not get updated when a page is changed from cornerstone. The last modified date specifically. It only changed if I went in to the Wordpress editor and clicked update.
Will this get fixed as part of the HTML update or is it something that needs to be looked at independently?

[charlie]

Probably early next year is the release timeline.

Saving in Cornerstone definitely updates the post modified date. It’s possible the plugin is checking for content changes, or there is some cache going on that isn’t cleared on a Cornerstone save.

[spedney]

I just checked in the litespeed settings and it’s not to do with caching there. I suspect it is something to do with RankMath and how they detect an update. *** Update: After checking with RankMath’s AI chatbot it recommended adding the following code to disable sitemap caching and that seemed to work:
add_filter( 'rank_math/sitemap/enable_caching', '__return_false' );