[12th May 2024] Interesting Things I Learnt This Week

1. How LLMs Work, Explained Without Math: This post by @miguelgrinberg explains LLMs working in simpler terms as well as defining some of the important LLM terminology. 

My Take: Now that all the initial hype around LLMs have kind of subsided, educating about the actual tech is taking a front seat. Its a good starting read for anyone trying to understand LLMs. I kind of loved the statement by the author "I wouldn't trust any workflow in which the LLM produces output that goes straight to end users without verification by a human.", which is pretty much on point here. 

 

2. 37 tips from a Senior Frontend Developer:  Ndeye Fatou Diop got inspired by  @abbeyperini's post and compiled his own list of tips. 

 My Take: These are some very sane tips which all software engineers should follow. These are not taught in schools but are valid none the less. All engineers learn these the hard way, but thanks to Ndeye you can learn is from his post there. 

3. Glance App: Glance is a a self-hosted dashboard. Users can add widgets to display things like RSS feeds, weather, and stock prices. Glance is lightweight and mobile-friendly.

My Take:  This is a very interesting app. It uses yaml for all configuration and is meant to be light weight with most of it written in go. It has a simple docker image which can be hosted anywhere, maybe even your own laptop. I think after iGoogle moved to graveyard, this is one of the best replacements I have seen for it.   

4. Google Removes RISC-V Support From Android: Google removed RISC-V support from the Android kernel. Google cited a lack of resources and the fragmented nature of the RISC-V ISA as reasons for dropping support. Others argue that Google is strategically blocking RISC-V to favor ARM processors. The future of RISC-V on Android is uncertain as of now.

 My Take:Google seems to be shifting focus towards AI and revenue-generating projects. While supporting RISC-V might have been strategic earlier, it appears to be a lower priority now. Some speculate this could be due to the ongoing US-China rivalry in technology. Regardless, Google's current actions suggest less emphasis on RISC-V compared to China. This could lead to China developing its own Linux version for RISC-V, or perhaps even the first widespread KDE phone.

 

5. Using an LLM for text compression: This article explores using a large language model (LLM) for text compression. The author achieves promising results by combining an LLM with arithmetic coding and a pre-processor for zlib. The experiment uses two LLMs, LLaMA-2-7B and LLaMA-3-8B, both quantized with a specific setting. The core idea is to provide the LLM with recent context and ask it to predict the 128 most likely following tokens. The longest predicted token is then chosen for compression. If none of the predictions exactly match the remaining text, the next character is encoded. The resulting sequence of integers is further compressed using ULEB128. The study shows that LLaMA-3 outperforms LLaMA-2 in most cases. The compression achieved is generally around 2 times better than gzip and at least 1.5 times better than Brotli.

My Take: This is an impressive piece of work! It reminds me of theories about DNA containing the entire blueprint for our bodies, and how it manages to compress that information so efficiently. The author did a fantastic job writing this up, but there's one key detail missing: the speed of encoding and decoding this information. The blog post doesn't mention it, but hopefully we can get it since the code is available on GitHub at https://github.com/arodland/llamacomp.

 

6.  Giant Compressed-Air Battery: This is an article about a company called Hydrostor that is building compressed-air energy storage facilities. The technology uses air, water, and a cavern to store energy for eight hours or more. The company believes this is a good solution for long-duration energy storage needs. They are building their first large-scale plant in Australia. Compressed air storage uses off-the-shelf equipment and could be cost-competitive with other options. However, the technology is still being proven.

 My Take: This is a not a novel idea, but I'm unconvinced it's an efficient way to store energy. As they said, it's unproven technology. I'd be happy to be wrong, though.