I spent the last week refactoring a single file on a single service of Luro to fix a production-only bug, which caused our server to fall over and took over 5+ hours to finish the weekly cronjob. Taking the bitter pill, we buckled down and did the work.

• 🧹 Redid the entire order of operations
• 😓 Scrutinized every for-loop and memory assignment
• 🪪 Added full JSDoc annotations with @typedef type declarations
• 📉 Reduced Cyclomatic Complexity 9 → 5
• 🧮 520 → 320 LoC (now fully documented)

If you’re interested in the results or the nitty gritty details, here’s the complete story…

Backstory

Luro has a Figma Analytics feature that scans your entire Figma team every week for your design system components. It’s a significant amount of work but was always a bit shaky under the hood. We went through a handful of rounds of defensive coding, fixing errors, handling exceptions, collecting garbage… but every so often we still had a bad weekend when the cronjob ran. Our solution was to scale up the server and that stopped the errors…

…until someone signed up with 2000+ components in their design system…

Oof. At this scale our problem went from bad to worse; servers were running out of memory and falling over and over, taking upwards of 5 hours to complete the cronjob. The hand grenade of technical debt had gone off. I volunteered for the job since I wrote the original code.

Redoing the entire order of operations

The biggest part of the work was redoing the entire order of operations. The high-level must-haves look like this:

1. We have to get Figma files
2. We have to get the components we care about (design system components)
3. We have to do some work to join those two concepts

It seems simple, but you’re looking for a lot of needles in some very big haystacks, which requires a lot of building up and filtering of objects and arrays.

In the original code we started by getting a list of all the Figma files for a team → fetched and parsed each file → and built up a big object in memory of all teams, files, components, projects, and nodes. Then we boiled the ocean to filter that object down to just the components in the design system. Then we ran what was leftover through some more cleaning and reformatting tasks so that it would be easier to consume on our front-end.

Peeling back the layers and understanding more of what was going on, I realized there was one big optimization we could make. Because Luro already knows about your components, we could start there and be more efficient with our file fetching (the expensive part). The refactor looks a lot like this.

Although process seems similar, it’s dramatically different. We first gather all the components → stub out our response (a list of component stats) → then during our (expensive) file loop routine we have a bouncer checking for components we care about:

• If the file doesn’t have components we care about, bail and do nothing.
• If the file has components we care about, do work and push data into the response.

Building the entire response first and pushing data into the response array was a lot more memory efficient and less CPU intensive than downloading everything → creating a big object → and boiling down a response. Files get processed in memory one at a time and we only do work when necessary. It also set us up for future improvements where we can do more async file work (the expensive part).

There’s a powerful metaphor here for modern front-end development about building a response first, then augmenting it, but I’m going to simmer on that for awhile.

Scrutinizing for every for-loop and memory assignment

The previous process was about reducing work we were doing at the macro-level, this next part is about reducing work at the micro-level.

If I have one criticism of JavaScript (actually, I have more) it’s that it makes it easy to unintentionally copy variables in memory. A classic example of this is Array#slice and Array#splice; one copies an array and one mutates an array and no one in the history of web development know which one does which. JavaScript also has a lot of syntactic sugar to make it easy to iterate through data. A map() here and a forEach() there are great until you find yourself looping through two thousand components in a quadruple nested loop.

Because for-loops and memory assignments have a propensity to get you into memory exhaustive situations (which is what we had), I scrutinized every line of code and tried to remove unnecessary assignments and looping operations wherever possible. Here’s a breakdown of the before and after:

	Before	After	% Improvement
= (assignment)	41	26	37.6%
… (spread)	5	2	60%
{ a,b } = c (destructure)	6	3	50%
if	8	7	12.5%
cond ? a : b (ternaries)	5	0	100%
for loops	5	5	0%
Object#keys	8	2	60%
Object#values	2	0	100%
Array#forEach	4	2	50%
Array#map	10	2	80%
Array#push	10	6	40%
TOTAL	104	55	48.2%

I’m happy with those results and more importantly it feels like there’s 48.2% less noise in the file. The code is also much easier to read and that’s thanks to this step and topics I’ll cover in the next three sections.

Annotating with JSDoc

Types are hype right now and while I groan against extra tooling they are helpful in scenarios where you’re passing around heaps of data. I could have used TypeScript but that (frankly) would have added more complexity to the project so I went with the much less intrusive JSDoc route.

You can enable JSDoc type inference checking in any file you’re working on in VS Code with the following one-liner (or in settings):

// @ts-check

Now red squiggles (type checking) show up all over the place. Next step was to create some custom JSDoc types with @typedef declarations.

/**
 * @typedef {Object} ComponentStat
 * @property {String} id - Component ID
 * @property {String} name - Component name
 * @property {String} sourceFile - SourceFile ID
 * @property {FigmaFile[]} files - Array of FigmaFile Obejects
 * @property {String[]} nodes - Array of node IDs
 * @property {Object} _count - Object of counts
 */

The final step was to start tying it all together by adding JSDoc blocks before our functions to set expectations on what parameters a function accepts and what it returns. Our main function, for example, takes an Object of options and returns a Promise that contains an array of our custom ComponentStat items that we created in the previous step.

/**
 * Get all components in files for all projects in a team and return Luro's Figma Analytics data
 * @param {Object} options - Options object
 * @param {String[]} options.figmaTeamIds - Array of Figma Team IDs
 * ...
 * @returns {Promise<ComponentStat[]>} - Array of components with usage counts
 */
export default async function getFigmaAnalytics(options) {
  ...
}

Establishing expectations between functions as well as tagging variables inside functions helped me spot places where I was creating similar data structures that could be combined or simplified. I don’t think I get the same high that TypeScript users get from fully typed applications, but a thin layer of JSDoc types was useful in this situation.

Reducing Lines of Code

Some organizations see lines of code as a productivity metric, but I see them as a liability metric. One by-product of refactoring was reducing our lines of code (LoC) from 520 → 320 (39% reduction). That’s a significant reduction considering it’s also fully typed and commented where it wasn’t before.

I will admit though that reducing LoC is a useless metric unless you have an agreement on what that means. Setting low LoC or kilobyte targets can be a trap where you end up reducing for reduction’s sake without acknowledging the trade-offs. I can golf code down in such a way that it become more obscure, more complex, and entirely unreadable. Cleverness can kill codebases.

To me, the goal for measuring LoC and keeping it low is that the end product is easier to read, easier to maintain, and less complex. Never lose sight of that goal. At 320 LoC the readability and maintainability is now absurdly high. But how could we be sure we made it less complex?

Measuring Cyclomatic Complexity

Ruby’s Flog gem first introduced me to the idea of “cyclomatic complexity” – a fancy way of measuring the number of nested for-loops and other farts you have in your codebase. No metric is perfect, but I wanted to measure complexity as a way to prove to myself (and my coworkers) that I was indeed making the code less complex and thus, hopefully, more memory efficient.

For our JavaScript project I was happy to learn ESLint has a built-in complexity scoring algorithm. To enable, add this to the top of your JavaScript files.

/* eslint complexity: ["error", 4] */

Adding this made a whole host of red squiggles (lint errors) show up. Through some tinkering with the error level up and down, I learned our original code had a complexity score of 9. A number I can work against! After a day’s worth of work, I checked and saw my score was at an 8. That was not the answer I wanted.

Senior engineer that I am, I used the ol’ comment-lines-out-until-the-errors-go-away method of debugging and identified dozens of places I could simplify the code. For example, the complexity linter hated Array#forEach and loved for...of loops. I think that’s stupid and overly pedantic. But looking at it now, it helped readability as well as exposed a weird bug with async functions inside a forEach loop.

As I mentioned above, the program was a lot of “find a needle in a haystack” type jobs. We had over a dozen casual uses of map() and Object.keys loops that were doing a lot of extra work. The code complexity exercise resulted in a final count of two needle-in-haystack searches broken out into their own functions, which cleaned up the main method considerably and increased our testability.

In the end I was able to reduce the complexity from 9 → 5. I get the code down to a 4, but the juice would not be worth the squeeze in regards to readability trade-offs.

Objectives and Key Results

How’d we do? We had a more critical need to relieve CPU and memory pressure on our application’s service. We approached that by refactoring the code, reducing the amount of code, and reducing complexity.

• 🧮 520 → 320 LoC (now fully documented)
• 📉 Cyclomatic Complexity 9 → 5

And the result was…

• ⏱️ 75% less CPU usage
• 🧠 58% memory reduction
• ⏲️ 5+ hours → 23 minutes

By all accounts our project was a success but we didn’t know if our refactor worked until we deployed it on production hardware. These fancy M1/2/3 Macbooks give a false sense of security compared to the reality of a $7/mo shared cloud server or even our more expensive $185/mo server. But measuring the cyclomatic complexity and reducing the lines of code gave us confidence that we managed to find a simpler solution that used less code.

It’s thrilling when a plan is successful but it would have been a worthwhile improvement even if it failed. That’s the nice thing about paying down technical debt is you’re (hopefully) leaving it better than when it started.

Next time I do this I’ll use flame charts, not only because they’re helpful but because they look good and sound cool.

Bless the maintainers

I enjoy maintenance work like this but it takes a lot of time, effort, brain power, as well as organizational trust to pull off an improvement project like this. Dedicating an entire engineer to a single file for an entire week is risky but as a result we have our feature is now less error prone, cheaper to run, and better documented; freeing up untold mental bandwidth.

I think collecting metrics are a nice way to grease the wheels of your organization and show progress on work that is essentially invisible. If I do a great job, no one will notice but it will be less noisy or potentially faster. A part of me wishes computers made a physical noise (beside Slack notifications) whenever they were having a bad time, then it’d be more obvious to everyone when code needs fixing. You wouldn’t have to convince anyone to pay down some technical debt because everyone would be yelling “Can you please make that fucking computer stop squeaking?”

A final note, it’s been interesting to do maintenance work in the context of Luro which is itself a tool that helps you maintain your product; keeping your front-end organized (via a design system), performant, and accessible. A lot of that is invisible work as well. The performance tab in Luro is already giving me ideas on what I can fix next.