How imports are handled in Pundle

This post is a part of the series “An Intro to Pundle”.

Imports are a way of sharing bindings between modules. Imports help keep the code modular and maintainable as the size of the codebase grows as well as sharing it with other people trying to solve similar problems (think npmjs).

In Javascript runtimes like V8 (heart of Chrome browser and Node.js) you can import JS files into JS files but you cannot for example import CSS files into JS files. Being able to import CSS into JS would help modularize the code further and help avoid any conflicts on the global css namespace.

Interoperability between asset types as different as JS and CSS (compared to say JS and WASM) requires a lot of decisions that are better done by the bundler instead of runtime. For example, whether to use the contents of the asset or export a path pointing at it because the size of the asset is too big (think 150kb+ of blob, wouldn’t make sense in a JS bundle).

While other module bundlers try to hard-code how the bridges work between types in the core, Pundle takes a different approach. Pundle’s file resolver allows users to specify completely arbitrary “format”s to use for each extension. Pundle enforces “chunk”s to have the same format throughout so when you import a css file or a golang file or any other non-js format file, It’ll be force loaded as js as to allow the transformers to provide “bridge” code.

While providing the bridge code the transformers can add the import as a chunk dependency and return path to it or return it as base64 blob etc. For css this lets us do interesting things like return the css modules object to the requiring module or insert css into a JSON blob that accepts HMR during development.

For example, here’s what happens when your import ./index.js imports an ./index.css

  • Resolve ./index.js relative to project root
  • Resolved to project/index.js with format js
  • Process project/index.js as js and scan for imports
  • Resolve ./index.css relative to ./index.js
  • Resolved to project/index.css with format css
  • Process project/index.css as js and scan for imports (notice how we’re processing as js)
    • If the file contains with .module before its extension like (.module.css), return its module mapping back to JS
    • If HMR and Development are turned on, put the raw contents as blob to create a style tag to be replaced on HMR later
      • Otherwise add project/index.css with format css as an external dependency of the file

This process depending on your configuration would either embed the css in JS file or add it as an external dependency (while also resolving its imports and adding them as dependencies recursively). This allows Pundle to maintain multiple versions of the same file with different formats. Handling imports like this allows for an unprecedented level of customizable interoperability between different formats.

How HMR in Pundle works

This post is a part of the series “An Intro to Pundle”.

HMR (Hot Module Replacement) is a way to apply changes to parts of the app while it’s still running allowing developers to quickly test new changes and be more productive. Webpack has a great introduction to HMR. I recommend you read it before reading through the rest of the post.

Here are some of the interesting things about Pundle’s implementation of HMR

Determining HMR server path:

Both Webpack and Parcel use WebSockets to deliver HMR to the clients. These can be a bit hard to develop / maintain because to handle WebSocket connections on the server side you have to either listen on a separate port for HMR (if source is on https, then you have to have https on ws too) or you have to provide the server object to the WebSocket libs (which means it cannot be an express middleware).

That leads to extra moving parts in the config, hmrHost or hmrPath. Pundle solves this by getting rid of WebSockets and using fetch() streaming instead. Fetch streaming works on normal HTTP requests so it can be done in an express middleware and does not need any third party library. For paths, Pundle uses document.currentScript + .hmr.pundle and then handles it as a route on the server.

As HMR is part of the development workflow, you don’t have to worry about supporting fetch streaming in older browsers as long as your own is up to date.

Compiling and applying changes

While Webpack uses a poll mechanism to determine if files have changed, Pundle makes the server do the heavy lifting. It writes HMR payloads to an internal list of streaming http requests. Pundle’s architecture allows it to generate individual files as temporary chunks, meaning that even if your bundle is 100mbs in size, only changed files would be transformed and sent to the client. This gives Pundle O(nFilesChanged) instead of O(nFilesTotal) time complexity for HMR.

Pundle also only regenerates a full bundle (ie joining all the transformed files together) only when it receives the next non-HMR request meaning that if you keep saving your files without refreshing, your CPU cycles won’t be wasted.

If you would like to read the source code of the HMR client (at the time of writing this post), You can hmr-client on Github

An Intro to Pundle

Pundle (Peaceful Bundle) is a next generation module bundler. It allows you to use Node.js style require statements in the browser and supports splitting out of the box with import() syntax. It uses worker processes to parallelize the work as much as possible, that combined with a file system transform cache delivers amazing performance on recurrent builds.

You can read how it works in this series of posts:

This series of posts is a Work-in-Progress and would be updated as Pundle furthers it’s path towards release.

Feminism vs Equality

Today is the 8th of March and is also the International Women’s day. While the media outlets use this day to promote a few women who did great things, I would like to pay respect and remember today the women who don’t get as much recognition but work very hard everyday to take care of their families and societies. These are our mothers, daughters, sisters and partners around the world.

It’s uplifting how the awareness against unjust treatment for women is raising around the world. We have come a long way from when women were burnt alive when their husbands died to when women are bosses in major institutions. But our struggle for justice is not over. True equality for all is possible only when we all try to do our part in ensuring equal opportunities and treatment for women around the world.

Countless movements around the world are pursuing this goal and among them is the Feminist movement. Like all the non-moderated public movements, it has a lot of flavors. In Feminism, there are people who want equality for all, and there are people who think women should have unfair advantage, both these groups living under the same umbrella.

Inequality against men is just as bad as inequality against women. Please stop using the word feminism for equality. If you believe in equality for all people regardless of gender, you are an equalist. Please do not associate yourself with people who are like the ones you are advocating against.

Thank You.

Why Reason?

Reason is a way to think properly while avoiding obstacles like bias, faith, emotions and beliefs. Reason is ultimately a tool for finding the truth. Instead of giving one person the authority to declare what is right and what is not, reason divides that authority among individuals.

Neither knowing the truth makes one reasonable, nor not knowing the truth makes one unreasonable. It’s the thought that “The person I am engaged in a discourse with might know something I do not. I might be wrong. I might learn something new.” As proven by Machine Learning in digital systems where neurons execute over datasets of millions, “On a large enough scale, quantity compensates for quality.” Thus, reason helps make societies where laws are fair, equal and enforced on all.

In a reasonable society, we take risks every time we make a statement. If the statement is verified, we gain trust of others or establish ourselves as liars otherwise. But the statements are not always verifiable, in which cases the liars would have a hard time convincing people of their occasional truth than truth tellers of their occasional lies.

The modern age of technology makes connection to the entire world incredibly easy. While it exposes us to people of vastly different cultures and beliefs, it also allows us to connect with millions of other people that think the way we do. People of the latter kind end up in an echo chamber that amplifies their thoughts ten folds. This pushes them further into the cave of ignorance and farther from the truth reason promises.

I have lost track of how many times I’ve seen a celebrity say things like “[Name] is a horrible person” without stating why they think that, which results in a backlash from supporters of the person referenced, which are again offensive statements without reason. This continues until the original poster claims the backlash is because of their race or gender and does still not explain why they said what they did.

Actions such as these are not only condescending but also very counter-productive. They do not encourage improvement while discouraging current behavior. The next time someone claims on Twitter, that someone else is a terrible person, don’t reply to them saying they are a terrible person for posting this. Instead, ask for their reasoning and proof so you can dismiss it or believe it based on your criteria of truth.

Gigabyte Z270 Gaming Ultra Hackintosh Build Guide

This build guide will walk you through the process of turning you Gigabyte’s high end Z270 Gaming Ultra Motherboard into a everyday-driver-usable Hackintosh machine. The motherboard works well in Hackintosh except that I haven’t been able to get the audio working with any other kext than Voodoo. A reminder to toss in an SSD into this build if you want it to be usable, macOS on HDDs can be slow.

I have already prepared an EFI folder so you don’t have to go through the complicated steps of figuring out which kexts to use and how compatible are some compared to the others. Here’s the list of all the kexts used in this build

  • ACPISensors.kext
  • CPUSensors.kext
  • FakeSMC.kext
  • GPUSensors.kext
  • IntelMausiEthernet.kext (for Mobo Ethernet)
  • LPCSensors.kext
  • USBInjectAll.kext (to make USB 3 work)
  • XHCI-200-series-injector.kext ( dependency of USBInjectAll )

While the sensors kexts are optional, they are highly recommended if you use programs such as iStatsMenu.

Setting up your Hackintosh

The first thing you’re gonna need is a USB disk to install macOS High Sierra on, You could follow the OSXDaily’s guides to downloading the public beta and making a bootable mac USB Drive.

You do NOT need to disable Vt-d or FastBoot as some guides recommend. If you have an AMD GPU (think RX series) you will have to set the initial display output to iGPU in your bios and plug in your monitor to the mobo display output. After macOS boots up you can plug it into the GPU display socket or switch the display input (if your monitor supports more than one).

Work is underway on solving this for AMD GPU users using the WhateverGreen.kext but I did not find it stable enough to use everyday as it’s display lags at around 30hz at the time of writing.

Next up is downloading the with preconfigured Clover that supports APFS and unzipping it into your USB.

Lastly reboot into the USB and proceed with macOS installation.

Update: The WhateverGreen.kext mentioned above has become stable enough to be a daily driver, you should include it in your kexts folder (EFI/CLOVER/kexts/Other/) if you are using an AMD GPU

How to be Happy

Happiness is a feeling. It’s not something we can eat or drink, it’s something we feel as a result of certain events. But even more important than that, Happiness is a result of the way we perceive the world.

Since the beginning of this year, I have chased happiness directly. I have tried different ways, while most of them have been unsuccessful, a small amount of them worked, or so I thought.

My initial experiment was with Pizza. Pizza used to make me happy, I used to have it once or twice a month but because I was chasing happiness this time, I started having it more than twice a week. Having a Pizza so often turned out to be unhealthy and it upset my stomach. The important thing here is that it lost it’s charm it used to have. I stopped getting excited for Pizza. I liked it’s taste but it did not bring me any joy.

My second experiment was with a local dish, the ones they served in my hometown were full of taste and joy. I thought I had found the answer to my question, the thing that would reliably make me happy. That’s until I moved to a different city and the dish they served there was terrible and not even close to the one I liked, thus putting an end to my reliable source of joy.

My third experiment was with shopping, I had this perception that shopping brings one joy. and it did bring me joy. That is until It stopped. The expensive shirts didn’t take long before losing their importance.

My last experiment so far has been with my Gaming PC. It’s one thing I love the most in this world, It’s as powerful as I had the means to make it and I get shivers in my body telling it to my friends, that’s how much I love it. While I love it, it’s stopped being the constant source of joy.

What I’ve concluded from the experiments mentioned is that when you depend on objects or people to be your source of happiness, you are going to be let down and you are going to get tired. Every external source of happiness has a limit.

So if you want to be happy:

  • Stop trying to be happy
  • Be your own best friend
  • Stop comparing yourself

I know this post doesn’t fit the usual theme of the blog, but I had to put it out there.