Blog
I’ve organized this section into several key categories, in order to make my posts more accessible. Enjoy!
Switch Transformers Mixture of Experts MOE
Paper's name: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (arXiv:2101.03961v3) 6/2022 Where are the lead Authors now? William (Liam) Fedus, Co-Founder of Periodic Labs Barret Zoph, rejoined OpenAI from Thinking...
Reinforcement Learning with Human Feedback (RLHF)
Paper's name: Training language models to follow instructions with human feedback (arXiv:2203.02155, Mar 2022) Where are the lead Authors now? Long Ouyang, Research Scientist at OpenAI Jeff Wu, AI researcher, Anthropic Xu Jiang, Founder, Light Robotics Diogo Almeida,...
LoRA, fine-tuning your model on a shoe-string from Microsoft Research
Paper's name: LORA: Low-Rank Adaptation of Large Language Models (arXiv:2106.09685, v2 Oct 2021) Where are the Authors now? Edward Hu Founding Partner, CTO (stealth AI startup) Yelong Shen, Principal Researcher, Microsoft Azure AI Phillip Wallis, Staff Research...
The paper that democratized LLMs for everyone
For another installment of my list of most influential AI papers, here's where the rubber hit the road, folks. Prior to distillation, running custom AI models was a costly and resource intensive process limited to those with deep pockets and access to large GPU...
RAG – Keeping your LLM up to date
Yet another installment of influential research papers that set the stage for the AI revolution. This week, it's all about a way to augment and enhance your LLM with up to date, context relevant data without needing to go through pre-training all over again. Title:...
The 2020 Research Paper that kicked off the AI Arms race?
In this second part on my series on influential AI papers, I'm going back to basics on the paper that kicked off the AI arms race, aka the GPT-3 paper Title: Language Models are Few-Shot Learners (Brown et. al., 2020) Where are the lead Authors now? Tom Brown -...
The 2017 Research Paper that made ChatGPT possible – did you know?
In this landmark 2017 paper, researchers at Google Brain and Deep Mind presented the Transformer Architecture, which allowed for massive parallelization of Machine Learning models and paved the way for modern Large Language Models (LLMs)
ITID and System Implementations
Understanding Information Technology Identity (ITID) is transforming how we approach system implementations.ITID describes how integral an information technology system is to a person's self-concept. Our research has identified eight distinct archetypes based on ITID,...
Mentoring – 20 Year Plan
One of the most effective tools that I have come across in my mentoring career has been the 20-year plan. If you haven't watched it already, please see my intro video on the 20-year plan here. Here's a 20-year template for you to download in Word (.docx) and PDF...
Online Customer Loyalty Programs
Purpose: To promote customer retention to our website, leading to increased revenue due to either of the following reasons: Customers buy more when they buy - Using up-selling techniques to include bundles or packages Customers buy more frequently - Promote increased...
Technology through the lens of Organizational Culture and Ancient Empires
This is an evolving area of interest in research. I'm interested in exploring the intersection of Technology, Culture and Ancient Empires; and how a parallel can be drawn between the cultural tenets of successful empires in ancient times and the technological...
2 Years and what next?
Hi all! Ook, still swinging around, not dead yet. Embarked on a new career at a hyper growth software company and busy building a global team. Lots of interesting projects and perspectives to share. Over the next few months, I will start cleaning up this blog a bit....
Upcoming Series on Cloud Companies
Hello all, Just wanted to provide a quick update on some research that I'm working on. There's been quite a bit of coverage in the media on various types of cloud solutions, from Infrastructure to platforms and also applications that sit on the cloud. While I've...
Update from the Chimp
Hello my banana loving legions of tech-loving simians. ook! Roadchimp here, back on the road (Istanbul). Since my last post, I've left the sunny English Isles, traveled around Asia for the summer and now on my way to Eastern Europe to deliver some workshops for a...
Cloud Architectures – Storage in the Cloud
Brief Cloud technology is deployed across a wide variety of industries and applications. The term 'Cloud' itself has become so widely prevalent that we've devised additional terms in an effort to describe what type of cloud we're talking about. What's your flavor?...
©2025 Sam Sena | All Rights Reserved