# Website Code Extractor

URL: https://sola.ysz.life/projects/code-extractor

> Any website's code as one ZIP

- Type: Personal project
- Period: Aug 2024
- Live: https://chromewebstore.google.com/detail/website-code-extractor/foppgeakfpkdghmmmflmblcidoofpohm
- Source: https://github.com/lyfe691/Website-Code-Extractor
- Stack: Chrome Extension, JSZip, HTML, CSS, JavaScript

Website Code Extractor is a one-button Chrome and Edge extension for saving a page to study how it's built. The ZIP it downloads opens straight from the folder, because the stylesheets, scripts and images its HTML references are saved next to it and linked by their new paths.

## A browser extension that saves a page and its files as a ZIP

[Website Code Extractor](https://chromewebstore.google.com/detail/website-code-extractor/foppgeakfpkdghmmmflmblcidoofpohm) is a Chrome and Edge extension that downloads the page in the current tab as one ZIP file. The archive holds the HTML and every stylesheet, script and image it references, in folders that mirror the servers they came from, and the HTML's links are rewritten to point at the saved copies. I built it in August 2024 to study how other sites are put together, and around 6,000 people use it now, according to the Chrome Web Store.

![The htmx.org home page with the extension's dark popup open at the top right; the Extract & Download button is pressed, reads Extracting… for a moment and returns](https://sola.ysz.life/projects/code-extractor/hero.webp)
*Saving htmx.org.*
[Video](https://sola.ysz.life/projects/code-extractor/hero.mp4)

### Saving a page

The popup has one button. Clicking it reads the page in the active tab, fetches every file the page links to and packs everything into a ZIP named after the site, here `htmx.org.zip`, and Chrome then asks where to save it. On htmx.org, building the archive took under a second.

Pages that can't be scripted, such as `chrome://` and `edge://` pages, `about:` pages and other extensions' pages, get an error in the popup instead.

### What the archive holds

The ZIP opens to one folder named after the site, with `index.html` at its top. Every other file sits under a folder named after the server it came from, followed by that server's own path. htmx.org's stylesheet, scripts and images land under `htmx.org/htmx.org/`, and the script for its GitHub star button under `buttons.github.io/`, so two servers' files with the same name can't collide. The copy of htmx.org held 53 files and 2.4 MB, and the site's Atom feed was one of them: the page links it from a `<link>` tag, which the extension collects like a stylesheet.

![The contents of htmx.org.zip, 53 files and 2.4 MB: a htmx.org folder with index.html, a buttons.github.io folder with buttons.js, and a second htmx.org folder with atom.xml, favicon.svg, css/site.css, 44 images and four scripts](https://sola.ysz.life/projects/code-extractor/zip-contents.webp)
*What the ZIP for htmx.org holds.*

### Opening the copy

Every stylesheet, script and image reference in the saved `index.html` points at the saved file, so the page opens straight from the folder with its styles and images in place. Links to other pages stay as they were, since the extension saves one page at a time.

![The saved htmx.org opened from the unzipped folder, scrolled to the quick start code block and the book section with the Hypermedia Systems cover image](https://sola.ysz.life/projects/code-extractor/saved-copy.webp)
*The saved htmx.org, opened from the folder.*

### How it's built

The extension is plain JavaScript on Manifest V3. A popup page with one script does all the work, and JSZip builds the archive.

#### Reading the page as it stands

The click runs a small function inside the tab through `chrome.scripting.executeScript`. Chrome serializes the function and runs it in the page, so it can't reach anything in the popup, and it hands back plain strings: the root element's `outerHTML` and the doctype. `outerHTML` leaves the doctype out, and a page saved without one opens in quirks mode, the legacy rendering mode browsers keep for pages written before the standards, so the doctype is serialized on its own and written back in front.

The saved HTML is therefore the page as it stands in the tab, after its scripts ran. A page that builds its markup in JavaScript is saved with the markup it built.

#### Rewriting the links

Back in the popup, `DOMParser` turns the HTML into a document that stays inert: its scripts don't run and its images don't load. The popup then walks every `<link href>`, `<script src>` and `<img src>`:

`popup.js`

```js
function resolveResourceLinks(doc, baseUrl) {
    const resources = [];
    const elements = doc.querySelectorAll('link[href], script[src], img[src]');
    elements.forEach(element => {
        let attr;
        if (element.tagName === 'LINK') {
            attr = 'href';
        } else if (element.tagName === 'IMG' || element.tagName === 'SCRIPT') {
            attr = 'src';
        } else {
            attr = element.hasAttribute('src') ? 'src' : 'href';
        }
        let url = element.getAttribute(attr);
        if (url) {
            if (!url.startsWith('http')) {
                url = new URL(url, baseUrl).href;
            }
            const urlObject = new URL(url);
            let path = urlObject.hostname + urlObject.pathname;
            path = path.startsWith('/') ? path.slice(1) : path;
            path = path.replace(/\/\//g, '/');
            resources.push({ url, path });
            element.setAttribute(attr, path);
        }
    });
    return resources;
}
```

Every reference is resolved against the tab's address first, so every form becomes a full URL. The hostname and path of that URL then do two jobs: they are the file's path inside the archive, and they replace the attribute's value in the HTML. For a page at `https://example.com/blog/post.html`:

| Written in the page        | Resolved to                        | Saved and linked as        |
| -------------------------- | ---------------------------------- | -------------------------- |
| `/css/site.css`            | `https://example.com/css/site.css` | `example.com/css/site.css` |
| `../img/logo.png`          | `https://example.com/img/logo.png` | `example.com/img/logo.png` |
| `//cdn.example.com/lib.js` | `https://cdn.example.com/lib.js`   | `cdn.example.com/lib.js`   |
| `/app.js?v=3`              | `https://example.com/app.js?v=3`   | `example.com/app.js`       |

The saved `index.html` sits at the top of the site's folder, so the new relative path leads to the saved file. Query strings are dropped along the way, since a cache-busting `?v=3` is not part of a file name.

Two tags from htmx.org as the server sent them, then as they read in the saved `index.html`, where `XMLSerializer` writes them back in XHTML style:

```text
<link rel="stylesheet" href="/css/site.css">
<script async defer src="https://buttons.github.io/buttons.js"></script>

<link rel="stylesheet" href="htmx.org/css/site.css" />
<script async="" defer="" src="buttons.github.io/buttons.js"></script>
```

#### Fetching from other servers

Fetching a stylesheet from another server is what CORS stops an ordinary web page from doing. Extension pages are exempt for the hosts listed in the manifest's `host_permissions`, and the manifest asks for four things:

| Permission                     | Used for                                                   |
| ------------------------------ | ---------------------------------------------------------- |
| `activeTab`                    | Reaching the open tab, and only after the click            |
| `scripting`                    | Running the function that reads its HTML                   |
| `downloads`                    | Saving the ZIP                                             |
| `http://*/*` and `https://*/*` | Fetching the page's files from whichever server holds them |

Chrome presents the last one at install time as permission to read and change data on every website, which is broad. The extension uses it for one thing: fetching, from the popup and only after a click, the files the open page already references.

The popup fetches them all at once with `Promise.all`. A fetch that fails is logged and skipped, so one missing image doesn't cost the whole archive. JSZip then builds the file, and `chrome.downloads.download` saves it, asking where first.

![The extension's page on the Chrome Web Store: the folder icon, a rating of 5.0 from 5 ratings, 6,000 users, and the popup as the first screenshot](https://sola.ysz.life/projects/code-extractor/store-listing-desktop.webp)
*The Chrome Web Store listing.*