A tale of SEO
(If you’re pressed for time, “The problem”, “next export” and “results” are the key informational sections, I hate burying the lede)
Why hello! It’s been awhile, hasn’t it?
Life has been pretty busy for me, with a good chunk of that time being spent on expanding my front end skillset in manners I had no idea I would be doing just a few years ago.
One of the most satisfying endeavors was started in November of 2020 and became “Medium blogpost worthy” when I made my little test bed a more meaningful site worthy of publishing. That site is a place for hosting my other hobby content, finances. If you’d like to check it out now, it’s FIcarious.com. However, that is not what this article is about, so let’s get on with it.
The problem
As it turns out, maintaining 7 years of technical debt as a (at most) 3 person team makes it difficult to make headway. That technical debt weighs heaviest when asked to move quickly to changing tides.
In November we were asked to put a heavy emphasis on our SEO, as the industry was receiving higher focus and we were starting to fall behind to our competitors.
At the time, we were entirely client side rendered. What that means is the entire content of our site was shipped along in Javascript and the page had to evaluate every line of JS prior to rendering anything. This was (and is) an extremely low cost of hosting approach that had worked very well for us over the years. However, what it fails to do is satisfy the SEO overlords who require content to be in the initial page load to be analyzed for meaningful information to the people searching the web.
In addition, the entirety of our application revolved around this approach to serving content to our users. This means it would be substantially costly in terms of time or resources to transition to a more modern approach of server side rendering.
With the need for SEO growth being time sensitive, coming up with a fast solution was more critical than the right long term solution.
In over my head
This was my first foray into SEO as somebody who had only ever worked on internal websites before. What was important? What wasn’t? How does Google really do it? How does this impact the customer experience? How am I to do this without rewriting years of work?
It was a pretty daunting task for me, but I was interested in the new challenge. Luckily, my company is no stranger to SEO and we had resources internal to the company to help answer a good number of questions. I looked into what it would take to server side render our most critical apps. The answer? A lot. We’re talking new CI/CD, repos, shared packages, et cetera.
Then I looked at the problem more critically and that’s where I started going awry. “If a client render has the content, then why don’t we host a client rendered page?” After all, it’s just React, right? A server side render works by evaluating a React tree to produce HTML, it then sends it to the client and React rehydrates itself using existing DOM structure. So, if I can provide the existing DOM structure and the React code that goes along with it, that’s all I care about, right?
Interestingly enough, this line of pondering is how we actually solved the problem! However, there was a middle step…
Look at me, I am the client now.
The “Captain Philips” meme comes to mind when discussing my “right in concept, horrible in practice” solution. I had the bright idea of mimicking a client during the build, and spitting out the results into an HTML file.
This was something I was able to achieve with little effort thanks to existing technology. Express & Chrome’s headless browser coupled with Puppeteer gave me the ability to spin up a Chrome browser in a docker container & target an Express server hosting the built files to render the page & access the resulting HTML. That HTML could then be written back to the file that the browser accessed, essentially replacing the HTML of the targeted render node.
In an overly simplistic tech stack, this actually works really quite well. Taking a React application that leverages CSS by way of CDN, rendering it, and injecting the HTML into your hosted site is actually just fine. It’s not the most performant, but it does satisfy the SEO piece substantially well.
The reason for this is that the React that generated it the first render is there for the second and would produce the same exact tree on the same mounting node. The result is that, on client load, React will re-establish the HTML exactly as it was rendered when you built it for hosting.
In terms of performance, it is less efficient than a server side render’s method of using rehydrate instead of render . This is because it has to remove the existing elements from the DOM and reinsert the newly constructed tree. For pages with sufficient content, that can be pretty detrimental.
However, I was able to get a little creative with that. I was able to control how the build was requested and therefore could add logic to look for a query param (or lack thereof) and choose to rehydrate, or to render.
What I had in the end was a system that “pre-rendered” our applications to produce a content rich first page load!
It took about two weeks to get this working correctly. I started to write up the docs & got ready to apply it to other pages and then…
Styled-components
Oh how close I was. It was pure happenstance that an upgrade to our page header included a transition away from the CSS based solution in place to leveraging the CSS-in-JS library called styled-components. This was a long time in the making, and we had implemented it in other pages. What I didn’t think about was how styled-components functioned.
Essentially, styled-components produces a style element in the head and produces a fully-formed stylesheet for the given elements in the experience. For a client side render, this is no problem at all. After all, the head is there, we can evaluate the tree on render, and update everything as we see fit. For server side rendering, it gets a bit trickier with having to extract the resulting element & putting it in your initial response, but doable.
For our pre-rendering solution? Well, we achieved putting the styles in the head no problem. It is just a verbatim output of what the client’s HTML would be. However, what was unexpected was that styled-components could not properly re-establish the relationship between the produced style element & the subsequent style element it produced during the rehydration. The end result was two fully fledged style elements that had hundreds (if not thousands given dynamic styling) rules in them. This meant every element had duplicate CSS rules applied to them. Even though the browser can resolve them to look right, it was non-performant. I tried nearly everything to resolve this, but it never produced a meaningfully performant site.
Re-enter NextJS
It was Christmas time. My wife flew home to see family and it was me and the dog for a week. What is a dev to do with all that free time?
I leaned on my go-to problem set to try out some new things, finances. I wanted to dig into NextJS and see how it worked under the covers to see if I could leverage that. renderToString , huh…that might work. Nope, didn’t produce the styles for the styled-components. So, how do they do that? ServerStyleSheet , ok, so styled-components has a solution for that. Neat. So I can take that and…oh, I need to insert it into the head. Now I have to traverse the DOM in Node or something to be able to insert it right. This doesn’t feel maintainable…
Having gotten frustrated enough with the pre-rendering attempts, I decided to focus on the finance side instead of the tech. I built out the net proceeds calculator which involved giving it some serverSideProps…only to read about getStaticProps . As it turns out, you can provide props at build time for Next. Digging into it a little bit further I found…the solution!
next export
As it turns out, NextJS already thought about my exact problem. I don’t need server side rendering. We don’t do anything that’s dynamic, so why bother hosting a node service & responding to HTTP requests if the content is going to always be the same no matter the request? If the server doesn’t have to re-evaluate the React tree on every call, we can cache the content indefinitely for every build, giving us the performance we want and the content rich response we need.
The only “cost” associated with this approach was to modify the few references to Javascript objects that do not exist on the server side (window, location, document namely). These were relatively simple things to do and I was able to get a prototype out the door in about a week.
This allowed us to leverage all of the features of server side rendering (i.e. providing a re-hydrate-able styled-components tree & main body content) and resulted in a static HTML file we could upload directly to S3, which was the existing architecture.
In addition, getStaticPaths allowed us to define dynamic routes and build out pages specific to our enumerable experiences. This meant we could define a single route for say, the list of states in the US, and produce all 50 states during the build.
Lastly, when converting pages from the client side rendering to pre-rendering solution, it set us up for the day where we do have the desire to implement server side rendering. The setup for each page is able to be fully leveraged in a server-rendering environment. This is proven, in a way, by the fact that you’re running server-side rendering while developing the code locally.
Limitations
The key limitation is in its static nature. Our core constraint is that we can’t produce dynamic content. While this feels like a very obvious statement, it rears its head in interesting ways.
When developing locally, you’re running in a server side rendered environment, and it can create an unreasonable expectation of behavior. This means you could approach a problem in a particular way locally, but it’ll never work in production.
It also means you have to reframe your thinking on certain problems, as well. We wanted to do an AB test on a script being included or not in the head, something easy enough to do with Server Side Rendering, but an impossibility in static rendering. That said, this was a problem that existed in our client side only solution, too! It’s just a bit more frustrating because you’re that close to having everything you need to accomplish the task!
Results!
Leveraging this pre-rendering solution, we were able to convert nearly all of our static client side pages over within 3 months (with the help of a new team being onboarded). This included a lot of rewriting of existing components into the new styled-components design system along the way, so had it been the sole focus, it likely could’ve been even faster.
As for SEO? We saw a steep improvement about 3 months after release, regaining the #1 spot in many of the places we had lost ground on. In addition, every experience touched improved over time, and many improved to the highest they have ever ranked in a saturated market.
Personally, the bigger win in my mind was around the core web vitals metrics. This was an unexpected side-effect of having an established document on load. For more on that, check out the blog related to how static rendering goes hand in hand with performance & provides a great platform for producing stellar sites!
