Are AMP pages really faster than HTML?
Short answer: NO
The Preface
The AMP Project is an open-source initiative aiming to make the web better for all. The project enables the creation of websites and ads that are consistently fast, beautiful and high-performing across devices and distribution platforms.
The quote above used to described shortly the AMP Project by Google on https://www.ampproject.org/ .
Many concerns about this technology were raised by web developers from all over the world despite it has been adopted already by many web platforms because of its simplicity.
Google is really pushing forward this technology and many articles about its benefits keep popping up here and there online. I have not really a strong opinion about it, and since I am a curious person, I wanted to do a bit of “fact checking”: I have tried to benchmark its performance against pure HTML and PWA pages.
I was also surprised to find online many resources describing the great advantages of such technology without actually showing any real data to support them.
The First Attempt
I have quickly setup a github repo and I tried running a simple lighthouse report on 3 different kind of HTML pages: AMP, PWA (with service workers and javascript) and pure HTML.
My benchmarks were run all locally on Chrome 68.0.3440.106 64bit via audit tab without fetching online assets ever (except for the core javascript of the AMP project).
The results of my first benchmark were clear: the pure HTML pages and the PWA ones have similar performance, but the AMP pages seem to take almost 3 times longer (3,5 seconds) to become interactive compared to a simple HTML file.
I published my results on Twitter but I got some critics about my method:
Basically some users complained that my tests were too simple and I should have tested AMP in complex application containing more than a simple <h1> DOM node. Challenge accepted :D
The Second Attempt
I agreed on the critics to my first benchmark so I decided to add a second one with a bit more DOM resources to fetch. I added 33 images 1 iframe and 1 video (mp4) to my pages to figure out how the 3 technologies would perform in that case.
I decided also that I could load the PWA assets lazily with 10 lines of javascript using the IntersectionObserver API.
Also this time it seemed that AMP pages were slower than the other 2 technologies. AMP pages required the injection of additional javascript files to handle videos and iframes. It’s worth to mention that the use of Javascript and lazy loading for the PWA demo made it de facto the most performant.
I have also noticed that AMP pages were not able to load the content of the embedded <iframe> due to an iframe origin policy issue.
The Conclusion
In my opinion AMP pages might provide many benefits to inexperienced developers or huge web teams that might struggle with third party ads or many assets to load without knowing how to do it properly. The syntax looks really nice and it’s based on custom elements! I am sure that the Google teams involved in this project know what they are doing and they are pushing it with their best intent but I am skeptic about the way AMP technology gets sold to the web community: it is simply slower than normal HTML.
All my tests are all on github https://github.com/GianlucaGuarini/amp-vs-pwa-vs-HTML,. Please feel free to check them by yourself and correct me if I did something wrong.
