Do You Really Want a Native Atmosphere?

In the event you’re an engineer, you’ve in all probability had the expertise of making an attempt to arrange or preserve a neighborhood growth surroundings. And whereas I could possibly be fallacious, I’d be prepared to wager that, as a rule, it was…not a terrific expertise. 

At a earlier firm, once I was simply beginning out writing code, a superb senior engineer spent a number of hours on a name with me summoning arcane CLI instructions to attempt to get my surroundings up and working so I might contribute to a mission. We lastly succeeded, however then it broke the next week in a brand new and complicated means when one thing modified. It wanted extra painful TLC earlier than I might get to the precise factor I wished to do, which was not, the truth is, sustaining a neighborhood growth surroundings, however transport code to ship worth to prospects.

At Assist Scout, our developer expertise workforce has made enormous progress in recent times to enhance what was already a fairly good native setup (you’ll hear extra about their vital work in future posts!). Nowadays, more often than not, my dev surroundings simply works, and when it doesn’t (often as a result of I haven’t up to date it just lately sufficient), I can typically determine why, unblock myself, and get again to work.

However then I’m a software program engineer, a sort of one that, relative to the inhabitants at massive, has a a lot increased tolerance for throwing spaghetti on the wall of an issue I don’t actually perceive. I scan by means of previous Stack Overflow threads for related morsels of data, work together with Docker andbrew andbash profiles, and, you understand, typically flip it off, strive one thing else, and switch it on once more (and repeatedly and once more) till one thing modifications. 

Not everybody needs to be a hacker

At most firms, although, there are a broader group of stakeholders — designers, product managers, QA specialists, assist workforce members — who would profit from the flexibility to check growth code however who realistically (and fairly!) are in all probability by no means going to need to take care of the trouble of configuring and sustaining a dev surroundings.

There are alternative ways to deal with the issue of getting code to those people earlier than it goes to prospects. One basic strategy is to ship the code to manufacturing behind a function flag, which you’ll be able to then allow selectively for the parents in your workforce who need to check it out. That is nice when it comes to simplicity of entry — these stakeholders simply have to log into particular accounts within the app they already use — but it surely actually limits agility and iteration as a result of every change requires a brand new Jira card, PR, evaluation, CI cross, and deployment earlier than the brand new code reveals up behind the function flag.

The recent new strategy to this downside is cloud-based environments, the shiny successors to staging servers. With this, you get the patron simplicity of testing in manufacturing (simply go to this URL and have at it!) with out truly having to ship to manufacturing, making iteration cycles means quicker. We’re experimenting with this paradigm and have had some good experiences, however then we get to a different downside, which is how exactly a neighborhood surroundings can replicate manufacturing. I believe everybody who’s had bother establishing a dev surroundings has in all probability additionally had the expertise of getting one thing working domestically after which, after merging and deploying it, discovering out that it doesn’t fairly work the identical means in manufacturing.

Standing on the shoulders of giants

I felt these two issues in my work and will perceive how they had been holding issues again, however, for some time, I didn’t actually know what to do about them. As I onboarded and obtained extra context about Assist Scout’s structure and engineering tradition (often by looking out Slack to strive to determine one thing I didn’t perceive), I’d often see references to utilization of an older Assist Scout mission known as ProxyPack, so in the future, led by curiosity, I made a decision to look into it.

From an archived GitHub repo, I realized that ProxyPack was an internally-developed Webpack plugin that was used to make it doable to run native growth JS code towards our manufacturing app’s backend (a PHP monolith supported by a community of Java providers). It wasn’t accessible to everybody and nonetheless required some native setup and terminal manipulation (together with establishing a spoofed SSL cert), however QA engineers I talked to who had been round when it was in use mentioned it was actually helpful for them when it comes to saving time and having extra confidence of their testing. Nonetheless, it was apparently troublesome to keep up (once more, you needed to take care of SSL certs!) and after the developer who constructed it left, it will definitely fell into obsolescence.

Be taught One thing Day

I’ll be trustworthy that my private pleasure in life will not be Webpack configuration (is it anybody’s?), so I didn’t contemplate making an attempt to revive ProxyPack itself. However the core concept was actually intriguing! Assist Scout encourages staff throughout the group to take common Be taught One thing Days the place they step away from their regular duties and concentrate on studying issues that can assist them (and by extension, Assist Scout) develop. This may take a wide range of varieties: taking programs, attending conferences, and spiking out prototypes and proofs of idea (POCs).

My favourite solution to study is by doing, so I all the time use today to hack on concepts. Whereas I’m not nice with Webpack, what I do truly know lots about is Chrome extensions. The truth is, a part of the rationale I work at Assist Scout is that in my earlier job, I constructed Chrome extensions on prime of it to customise the app to my workforce’s particular wants. So the following time I had a Be taught One thing Day I began to noodle round with an concept: May I get the identical end-user final result of ProxyPack — the flexibility to simply check dev code towards manufacturing — by means of an extension?

Bundles of pleasure

My inspiration got here once I was trying on the Community tab to look at an API response in manufacturing towards the one I used to be getting in my native surroundings. Whereas I used to be scanning by means of the checklist, I by chance clicked on one of many JavaScript bundle requests fairly than the API response I used to be on the lookout for. Like many apps, we retailer our JS code in a distributed CDN for the quickest doable entry by customers irrespective of the place they’re.

Let’s say the bundle URL for one of many SPAs that make up our app seems to be like this:

An concept began to develop in my thoughts. I switched to the opposite browser the place I used to be working the native surroundings. The native dev surroundings builds the JS code and runs it on a easynginx server that’s utilized in lieu of the CDN.

I had a eureka second — what if, whereas utilizing the manufacturing app, I might programmatically intercept requests to the manufacturing variations of the bundles and redirect them to the native variations of the identical information? 

Declaration of independence

Chrome extensions have a wide range of superpowers not granted to plain net apps. One of the highly effective of these is the flexibility to work together with HTTP requests made in a consumer’s browser. The normal mechanism for doing so was thewebRequest API, which allow you to register a callback for HTTP requests and allowed you to eavesdrop on them or, with theblocking possibility, intercept and modify them (URL/headers/request physique) whereas they’re in flight.

ThewebRequest API would have made implementing my concept very simple, and I began with that as a proof of idea, however then I realized that theblocking possibility forwebRequest is within the technique of being deprecated. Nonetheless, Chrome has supplied a successor towebRequest, thedeclarativeNetRequest API. 

This API is far tighter when it comes to what it permits you to do to requests (and presumably its declarative nature additionally makes it simpler for Chrome’s app reviewers to automate scans for dangerous extension conduct). Slightly than having full management to imperatively modify the requests your self in your extension code withwebRequest, you register guidelines (which explicit requests to do issues to) and actions (what issues to do) in response to a restricted JSON schema after which the browser itself handles all of that externally (and extra securely) as a substitute of inside your extension code.

Request…intercepted

On the root of a Chrome extension’s code is a manifest requesting the permissions your extension wants and itemizing the situation of varied belongings in your app so it is aware of the place to run your code. Right here’s a minimal manifest for our use case, claiming thedeclarativeNetRequest permission and directing the extension to the place our rule is registered:

{
  "identify": "native proxy experiment",
  "model": "1.0",
  "manifest_version": 3,
  "description": "Proxies hs-app-ui JS requests to hs-stack bundles",
  "host_permissions": ["<all_urls>"],
  "declarative_net_request": {
    "rule_resources": [
      {
        "id": "proxy",
        "enabled": true,
        "path": "proxyRule.json"
      }
    ]
  },
  "permissions": [
    "declarativeNetRequest",
  ]
}

I began out by hardcoding a rule for one file:

{
    "id": 1,
    "precedence": 1,
    "motion": {
      "kind": "redirect",
      "redirect": {
        "url": "
      }
    },
    "situation": {
      "urlFilter": "https://helpscoutblog.cloudfront.internet/frontend/inbox-settings.js",
    }
}

As you possibly can see, this rule has aurlFilter situation in order that when a request is made for the manufacturing bundle, it redirects it to the native model. I activated the extension, made a change, and refreshed the web page, and it labored — I might see my native dev code within the manufacturing app!

We cut up our bundles by route to cut back pointless code loading, although, and I didn’t need to need to hardcode all of them after which have to replace the extension any time we added a brand new one.declarativeNetRequest has an possibility to assist deal with that, fortunately — you’re in a position to make use of regexes each for setting situations and for modifying the ensuing URL. That sounds scary (OK, regexes all the time sound scary to me!), but it surely’s actually not that dangerous. Right here’s the up to date model:

{
    "id":1,
    "precedence": 1,
    "motion": {
      "kind": 'redirect',
      "redirect": {
        "regexSubstitution": 'https://localenv.helpscout.com/frontend/1',
      },
    },
    "situation": {
      "regexFilter": 'https://helpscoutblog.cloudfront.internet/frontend/(.*)'',
    },
}

What’s occurring right here is that within thesituation, we’re capturing the filename with(.*) then utilizing the numbered capturing group1 to append that to the top of the URL we’re redirecting to. With that setup, I might transfer by means of varied apps in Assist Scout, make modifications domestically, refresh the tab, and see my modifications “in manufacturing.” I shared this POC (which, in tribute to ProxyPack, I named ProxyPal) with the workforce and began utilizing it in my very own work; a number of engineers and QA people additionally reported discovering it useful.

Increasing the consumer base

I used to be pleased with what I had constructed, but it surely didn’t absolutely deal with my supposed use case. It helped for extra technical QA people (and was additionally nice for engineers testing bug fixes in growth towards the manufacturing app and information the place the bug was occurring), however you continue to wanted to keep up a neighborhood git repo and run the dev server, so it wasn’t as universally accessible as I wished it to be.

Fortunately, my teammate Maxi Ferreira is a Cloudflare skilled (in addition to a terrific e-newsletter author), and he helped me use that service to make ProxyPal out there to everybody on the workforce. Right here’s what we did:

• On push to a PR department, we constructed a duplicate of the JS belongings and revealed it to Cloudflare Pages.

• Utilizing Cloudflare’s superior built-in GitHub integration, each PR contains an auto-generated remark with a hyperlink to that construct.

• The extension does a bit additional magic utilizing a content material script that injects a button subsequent to the Cloudflare remark:

import './github.css';
const getPrTitle = () => {
  return doc.querySelector('.gh-header-title').innerText;
};
operate handleLinkInjection() {
  
  operate injectLink(aTag) {
    
    if (aTag.href.contains('pages.dev') && aTag.href !== 'https://pages.dev/') {
      
      var newLink = doc.createElement('button');
      const prTitle = getPrTitle();
      newLink.textContent = 'ProxyPal'; 
      newLink.className = 'proxyPalButton';
      newLink.onclick = () => {
        chrome.runtime.sendMessage({
          kind: 'SET_ACTIVATION_STATUS',
          worth: {
            standing: true,
            envUrl: aTag.href,
            envDescription: prTitle,
          },
        });
        alert(`Now proxying to ${prTitle} at ${aTag.href}`);
      };
      
      aTag.parentNode.insertBefore(newLink, aTag.nextSibling);
    }
  }
  
  doc.querySelectorAll('a').forEach(injectLink);
  
  var observer = new MutationObserver(operate (mutations) {
    mutations.forEach(operate (mutation) {
      mutation.addedNodes.forEach(operate (node) {
        
        if (node.nodeType === 1 && node.tagName === 'A') {
          injectLink(node);
        }
      });
    });
  });
  
  observer.observe(doc.physique, {
    childList: true,
    subtree: true,
  });
}

operate handleNavigation() {
  
  if (window.myMutationObserver) {
    window.myMutationObserver.disconnect();
  }
  
  handleLinkInjection();
}

handleLinkInjection();
let lastUrl = location.href;
setInterval(() => {
  const currentUrl = location.href;
  if (currentUrl !== lastUrl) {
    lastUrl = currentUrl;
    handleNavigation();
  }
}, 1000);

chrome.runtime.onMessage.addListener((request) => {
  swap (request.kind) {
    case 'SET_ACTIVATION_STATUS': {
      const { standing, remoteUrl } = request.worth;
      if (standing) {
        chrome.declarativeNetRequest.updateDynamicRules({
          removeRuleIds: [PROXY_ID],
          addRules: [
            {
              id: PROXY_ID,
              priority: 1,
              action: {
                type: 'redirect',
                redirect: {
                  regexSubstitution: `${remoteUrl}/1`,
                },
              },
              condition: {
                regexFilter: "https://helpscoutblog.cloudfront.net/frontend/(.*)",
              },
            },
          ],
        });
      }
    }
  }
});