This case study delves into how Kiwix, a non-profit organization, leverages Progressive Web App technology and the File System Access API to empower users to download and store massive Internet archives for offline use. Dive into the technical implementation of the code dealing with the Origin Private File System (OPFS), a new browser feature within the Kiwix PWA that enhances file management, providing improved access to archives without permission prompts.
About Kiwix
Kiwix is a Switzerland-based non-profit organization dedicated to making knowledge accessible to people worldwide. Founded over 30 years ago, their mission is to bridge the gap for those with limited or no internet access by developing open-source apps and content that can be downloaded and stored offline. With this goal in mind, Kiwix has created an ecosystem of tools that enable users to store gigabytes of data from the internet for offline use.
ZIM Archives: A Compressed Solution
Kiwix uses highly efficient Zstandard (ZSTD) compression to store HTML, JavaScript, and CSS files, while images are converted to compressed WebP format. Each ZIM archive includes a URL and title index, allowing users to access content quickly. The entirety of Wikipedia in English can be compressed into 97 GB after conversion to the ZIM format, making it possible for users to store vast amounts of knowledge on their devices.
Native Apps and PWAs
Kiwix offers native apps targeting desktop (Windows/Linux/macOS) and mobile (iOS/Android) usage. This case study focuses on the Progressive Web App (PWA) that aims to provide a universal and simple solution for any device with a modern browser.
Challenges in Developing Offline-First PWAs
Kiwix users have diverse needs, and the organization has little control over the devices and operating systems they use. Some devices may be slow or outdated, especially in low-income areas. To overcome these challenges, Kiwix developers built a dedicated PWA version of Kiwix JS, incorporating OS integrations that allow native-like capabilities, such as offline usage, installation, file handling, and file system access.
The Role of Service Worker API
Offline-first PWAs are extremely lightweight, making them perfect for contexts with intermittent or expensive mobile internet. The Service Worker API and Cache API enable apps to act as a server, intercepting Fetch Requests from the main document or article being viewed, and redirecting them to the JS backend to extract and construct a Response from the ZIM archive.
Storage: A Key Concern
Given the large size of ZIM archives, storage and access to it are critical concerns for Kiwix developers. Users can download content in-app when internet is available for later offline use or transfer files between devices using USB sticks or portable hard drives. Kiwix JS and PWA must support these various methods of accessing content from arbitrary user-accessible locations.
Unlocking File System Access
The File API initially enabled Kiwix JS to read enormous archives, even on low-memory devices. This API is universally supported in any browser, acting as a universal fallback when newer APIs are not supported. By defining an input element in HTML, Kiwix can select and access large archive files.
Target Keyword: Swift app development