All Articles

20 min read • By Kyle Truong • Published 11 Feb 2020

Lessons Learned Building My First Shopify App

Finally, after six months of full time development mixed in with ample amounts of thumb-twiddling and compulsive distractions, I’ve finished building my first shopify app and it’s been released on the Shopify app store. I feel a tremendous amount of energy within this ecosystem and I want to share with you my experiences and lessons learned building my first Shopify app from the gritty technical details to high level takeaways.

Lessons on the Shopify API

Starting out, the documentation and tutorials for building Shopify apps do a fantastic job of guiding you on how to build your app, but where I found bumps on the way were areas that dealt with modifying the storefront. In most Shopify apps there will be admin and storefront sections, the former is the area exposed to merchants and the latter is the area exposed to the customers of the merchant.

With the admin sections, both the REST admin API and GraphQL admin API are more than comprehensive in their documentation. But when it came time for me to start creating the storefront section of my app, I found the options plenty but the documentation either sparse or lacking context.

To start, there are script tags, app proxies, asset APIs, and the Storefront API (and app snippets and sections too, but these are still only in developer preview mode at the time of writing this). I started with the Storefront API because I figured if I wanted to modify the storefront then that’s what I’d use. It turned out this was the wrong choice, at least I think it was. Honestly, I’m still not sure what to use the Storefront API for but it doesn’t seem like it was made for my scenario (displaying a widget on the storefront).

“The Storefront API gives you full creative control to build customized purchasing experiences for your customers.” — https://help.shopify.com/en/api/storefront-api

Note: This quote came from their old documentation at the time of writing this article.

Perhaps it was clear but I often fall victim to reading what I want to read instead of what’s actually there, so this was confusing for me because I wanted to modify the storefront but I wanted nothing to do with customized purchasing experiences. The lesson here is that the Storefront API is not the tool to use for adding widgets to your merchants’ storefronts, or if it is, then I’m sorry for adding to the confusion.

Script Tags

I found the Script Tag much simpler to understand because all it does is allow you to inject JavaScript into the shop’s storefront or order status page of checkout. I found this sentiment echoed through the Shopify community forums as I searched for answers on what the best way to modify storefronts was. Unlike the asset API, Script tags are automatically removed from the store upon uninstall and don’t directly mess with your merchants’ custom themes. To start, all you have to do is create a JavaScript file and use the ScriptTag API to install it, usually upon installation of your app:

// app.middleware.js

const buildScriptTagSrcUrl = (): string => {
 return process.env.SCRIPT_TAG_URL || '';
};

export const initializeApp = async (ctx) => {
 ...
 await createScriptTag(
   buildScriptTagSrcUrl(),
   SCRIPT_TAG_DISPLAY_SCOPE.ONLINE_STORE,
   shop,
 )
 ...
}

But of course, this is only the start:

  • How do I use my favourite libraries within the script?
  • How do I locate a specific area within the theme to inject my widget?
  • And how do I make sure the requests from the script are authenticated within my app, and what about CORS?

Books have been written on these topics, but for the sake of brevity I’ll try my best to keep it concise.

There’s no magic to the script, it literally is just JavaScript. No fancy React or Polaris here unless you manually load them within the script, but be careful if you do because every library added impacts performance for your merchants’ stores. If you still wish to add a library, I suggest loading the script as shown in the docs:

https://help.shopify.com/en/api/guides/modifying-online-store/use-javascript-responsibly

I decided to go vanilla JS rather than jQuery to keep the bundle size small, and I’ve just got to say I’m now so much more grateful for all the advances made with client-side tooling, especially with modules, and components, and npm packages. The majority of my script contained functions that located, created, and modified DOM nodes, but there were also sections that contained logic related configuration and data transformation, and it was all stuffed into one 1000-line file.

The most troubling part for me was discovering that there was no automated way to consistently inject my widget where I wanted. Each theme has a custom HTML layout and your widget better be prepared to accomodate each one or else it just won’t load. Shopify expects the app to work at least with all the free themes provided, but if it’s a custom theme it will likely require additional support on your end. On the bright side, the upcoming app snippet and app section API looks promising in alleviating these pain points.

My script needed to make requests back to my app in order to fetch store-specific data, and I found most people either used AJAX or jsonp. It was my first time hearing about jsonp, an alternative method of loading data. It requests data through <script> tags, and requests are fine-tuned by callbacks in the form of string parameters within the src of the <script>. It’s really weird. It was created to bypass the problems posed by CORS–if your script was hosted on a CDN and loaded in a storefront then the request coming from the script would be of a different origin than your server, which is a no-go when sharing data.

Either way works, but if you want to go the AJAX route I’d recommend pairing your requests up with Shopify App Proxies in order to improve your security. With AJAX, the simplest way to bypass the CORS issue is to accept requests from all domains, but then the problem becomes one of security–how do I filter out malicious requests?

Adding in App Proxies

App Proxies allow you to have a dedicated route on the storefront that proxies all requests to it to an endpoint of your choice. From this endpoint, you can return liquid, html, and json too. The reason I recommend this tool is because all requests sent to the app proxy from Shopify carry an hmac that can be verified only by the shared secret key known to you and Shopify. Now with app proxies, requests can be made from the script tag using fetch to your app proxy, where the request can be authenticated and store-specific data can be returned as a response in the form of json.

Lessons on Software Development

By far, one of the best things about running my own software business is the freedom to use any library I wish. At the same time, this lack of restraint is probably one of the main reasons my app is failing, business-wise. I spent so much time toying around with these new libraries that I neglected every other aspect. I still had fun as I got to use tools I never got to use in production before because they were deemed too new or risky–tools like Kubernetes, Graphql, and design systems.

DevOps and Kubernetes (K8s)

Every app needs some sort of pipeline to take code from their local machines to production. Just to set some context and settle semantics, I’m calling the area of writing code as development and the area of maintaining your infrastructure in production as operations, and the process that links the two together as DevOps. Without a strong and clear DevOps pipeline pushing to production is inefficient at best and nightmare-inducing at worst.

I never liked the idea of having different deployment processes for different environments and platforms. Sure, it would be simpler to just spin something up for free on Heroku and then move to AWS when needed, but each pivot creates unnecessary friction in learning platform-specific nuances, and no platform is guaranteed to stay suitable as both your needs and the platform change over time. I’m looking for one process I can use for every environment, for every platform, for every amount of traffic, and for every new project I spin up. And it’d be nice if it was free and had a thriving community too.

Kubernetes and its surrounding ecosystem of tooling has been of interest to me for a while and I wanted to try it out. In theory, I would be able to describe my infrastructure in code with K8s resource files written in YAML, and I would be able to use that same code on local machines for development (with minikube and skaffold), on staging, and in production. Deployments and rollbacks would follow the same processes using Helm, and any variation in environment-specific configuration can be declaratively set in YAML files.

My initial plan was to have everything under K8s–the load balancer, the app, and the database. What I quickly learned was that managing state in K8s was, by far, the trickiest part. I looked a bit into stateful sets, volumes, and CNI plugins but found myself completely lost. Databases are hard enough, but putting databases in a container orchestrator is a whole other level. I went with a managed database instead.

Using K8s for development proved more challenging than I expected too. I got it running once using skaffold but updates took too long between code changes due to my poorly written Dockerfiles and there was an odd interaction between my mac, minikube, skaffold, and the ingress that produced what looked like some caching bug. I gave up on using K8s in development at this point and resorted to the trusty terminal.

Still, when it came to production deployments and rollbacks nothing came close. Last week, I updated my app to accomodate the new third-party cookie settings for chrome:

https://blog.chromium.org/2019/10/developers-get-ready-for-new.html

One tricky thing about developing Shopify apps is that all initial requests come from Shopify and if you want to test with Shopify then you have to configure your app within the Shopify admin panel, meaning it’s a PITA to switch environments because you have to manually switch all your URLs and proxy paths. I didn’t want to do that, and seeing that I had zero users in production, I decided to do development in production.

Say what you will about my process, but I never before felt so secure and in control of my deployments and rollbacks as I updated my cluster over 40 times during the night with the same commands:

Helm list

>> NAME          	NAMESPACE	REVISION	UPDATED                             	STATUS  	>> CHART    	APP VERSION
>> ccs-1580083321	default  	48      	2020-02-11 21:24:00.889075 -0500 EST	>> deployed	ccs-0.1.0	1.16.0

Helm upgrade ccs-1580083321 ccs
Helm rollback ccs-1580083321

and every single update did exactly as I expected:

Helm and Kubernetes in action on the command line

(I like to debug by compulsively re-refreshing after making one change)

I don’t understand how Kubernetes does the things it does and I probably never will, but I was and still am surprised at all it can do. From the start, I knew K8s was unnecessary for an app as simple as mine, and it costed the most in terms of development time, but I’m glad I went down this route because of how valuable a tool K8s is. Having my infrastructure handled by K8s enables a great amount of scaling, flexibility, and consistency for this project and for every project from here on out.

GraphQL

I think it’s no secret that GraphQL is growing and that Shopify is betting big on GraphQL, so I wanted to try it out too. The biggest divergences I’ve found with using GraphQL happen to be on the frontend when picking a library to use. The big two players seem to be Apollo and Relay. Both seem to be comprehensive libraries but also require a large investment into their ecosystem and tooling.

All I wanted was a library that executed GraphQL requests with plain GraphQL queries but these libraries were offering so much more, including state management. I decided to go with the simplest GraphQL client implementation, fetch:

// shared.ts

export const graphqlRequest = async <T>(
 query: string,
 variables = {},
): Promise<T> => {
 const headers = {
   ...buildContentTypeHeader('application/json'),
 };

 const body = JSON.stringify({
   query,
   variables: isNil(variables) || isEmpty(variables) ? null : variables,
 });

 return await fetch(createGraphqlUrl(), {
   method: 'POST',
   headers,
   body,
   credentials: 'same-origin',
 })
   .then((responseStream: any) => {
     return responseStream.json();
   })
   ...
};

...

// app.epic.ts

Const appDataInitializationQuery = `
  Query AppDataInitializationQuery {
    Products {
       ...
`

export const appDataInitializationEpic = (
  ...
): Observable<StoreAction> =>
 action$.pipe(
   ...
   asyncRequest(APP_DATA_INITIALIZATION_ASYNC, (action: StoreAction) => {
    return graphqlRequest(appDataInitializationQuery);
   }),
 );

I also found that redux was still valuable as a state management tool thanks to its pluggable middlewares and simplicity in debugging, though, if you’re looking for a powerful but lightweight solution that can work with just React and hooks, I’d highly recommend looking into urql:

https://github.com/FormidableLabs/urql

On the backend, I found koa-graphql paired with graphql-tools sufficient to build my graphql type definitions and resolvers. They look something like this:

// cross-sell.graphql.ts

export const crossSellQueryTypeDefinitions = `
  crossSell(crossSellId: Int): CrossSell
  productCrossSells(productId: String): [CrossSell!]
`;

export const crossSellTypeDefinitions = `
  type CrossSell {
      displaysProductPrice: Boolean
      displaysProductTitle: Boolean
      id: Int
      productId: String
      shopId: Int
      title: String
      type: String
      typeValue: String
      customCrossSellProductIds: [String!]
    }
  }
`;

export const buildCrossSellQueryResolvers = () => {
  return {
    crossSell: async (
      _: any,
      args: { crossSellId: number },
      ctx: AppContext,
    ): Promise<CrossSell> => {
      const { crossSellId } = args;

      const crossSell = (await ctx.state.dataLoaders.crossSell
        .load(crossSellId)
        .catch(error => {
          logError(error, ERROR_MESSAGE.CROSS_SELL_FETCH_FAILED, ctx);
          ctx.throw(500, getStatusText(500));
        })) as CrossSell;

      return crossSell;
    },

export const buildCrossSellResolvers = () => {
 return {
   CrossSell: {
     id: async (
       root: CrossSell,
       args: any,
       ctx: AppContext,
     ): Promise<number> => {
       return root.id;
     },

     shopId: async (
       root: CrossSell,
       args: any,
       ctx: AppContext,
     ): Promise<number> => {
       return root.shopId;
     },

The biggest issue I found here was the n + 1 problem, where a request for a few fields on the same entity may end up creating an unreasonable amount of queries to the database. It’s a common GraphQL issue and can be solved with help from the dataloader library. More information on this problem and its solution can be found here:

https://engineering.shopify.com/blogs/engineering/solving-the-n-1-problem-for-graphql-through-batching

Overall, I enjoyed using GraphQL and I see there are some benefits of adopting it over REST APIs, my favourites being that backend data can now typed and structured according to how its used on the frontend, and that that same data can be sliced and diced in different permutations without having to edit the backend code. I’d describe my experiences with GraphQL as “fun”.

Lessons on Design

Most Shopify apps are unique in that they have two types of users: the first being the merchant and the second being the merchants’ customers. As you design your app, you’ll have to account for both types of users. I made the mistake early on of tunnel-visioning my design efforts for the merchant, and so when it came time to design for the customers I found many details were unaccounted for in the merchant portion, ultimately leading to a complete re-design of my app.

Design requires a different mode of thinking compared to coding, and when that fact isn’t respected time is wasted. I’m far more proficient with development than I am with design so what often happened was that my development would progress past design and then I would have to simultaneously juggle design and development, resulting in inefficiencies, frustration, and poor results. Take the time to design. Make sure you’ve understood the user experience and flow of the app, created the necessary wireframes, accounted for different interaction states, and gathered all the assets required before you start developing that piece of the app.

Also, Shopify’s design system, Polaris, is incredible and should be heavily considered when designing and developing Shopify apps. Polaris saved me an unimaginable amount time by cutting out all the planning and design work required to create a consistent feel throughout the app. It’s so much easier to develop an app when all the components, colors, and restraints have already been designed and exposed in the form of style guides, design kits, documentation, and React components. Paired with the collaborative interface design tool, Figma, I was able to create fast and accurate wireframes and flows that I used as a foundation for while developing.

Lessons on Business

The idea for my app came up in the spring of 2019 while I was still working full-time. I looked at the ecosystem and saw there was a gap and thought I could fill it by creating a better designed app for cheaper. What ended up happening was I spent all my time learning shiny new tech, and by the time I launched, I learned that the gap I saw was not significant enough to pull away traffic from my competitors, who had now also improved their app to the point mine was now obsolete.

First and foremost, a business needs to solve for a real problem in the ecosystem. Second, a business needs to create the right solution. Everything else is secondary. I saw a problem but I didn’t verify it, I relied on the fact that my competitors were succeeding as proof that my problem was real. The solution I created was also vague and untested, so there was no way to know for sure if I had the right solution. I think the best way to verify my two requirements would have been quick iterations.

I find it funny that I used to work for a company that championed an agile process but now I see myself repeating the very same mistakes I saw our clients making–the mistakes we worked overtime to bring to light and fix. Maybe I’m just stubborn and need to experience failures first-hand before I learn.

Anyways, it would have been far better to fail hard and fast rather than keep developing for things I only thought were useful. I realized that I may not be at the point where I see what needs to be done, but by failing fast I know what shouldn’t be done, and sometimes that’s just as good. The farther I stray from the things that shouldn’t be done the closer I get to the things that should be done.

Next Steps

I’m thrilled to have finally developed my first Shopify app. I’ve still got a sweet tooth for shiny new tech that I’ll be indulging in for my next project even if it’s at the cost of business. I look forward to developing many more apps and sharing the lessons learned.