Articles, analysis, and field notes filed under Article.
Article
AI Infrastructure Where We're Going, We NEED Roads Networking is not the supporting act in AI clusters. It is the system itself. Last week in London, I stepped out of Sainsbury’s with a couple of Jellycats for my...
Article
AMD's challenge is architectural: their "world size" is limited to 8 GPUs per node, while Nvidia can interconnect 72. AMD's solution won't arrive until 2027, leaving Nvidia with a commanding lead for training and large-scale inference.
Article
AMD MI355X Analysis Strong Node-Level Inference, but Not Yet Rack-Scale 288 GB HBM3E Memory 8 TB/s Bandwidth 1.4 kW Power Draw Quick Navigation 1. Overview 2. Performance Analysis 3. Memory Advantage 4. Interconnect...
Article
A recent Synergy Research Group report has revealed a striking statistic that should give every infrastructure professional pause: hyperscalers—AWS, Azure, Google, Meta, and their peers—now account for 44% of the world's total data center capacity. More importantly, this figure is projected to surge
Article
During my deployment engineering days at Amazon, we were in the middle of what felt like controlled chaos. New AWS regions launching monthly, services expanding globally, data centers coming online faster than anyone thought possible.
Article
The world of large‑language‑model inference moves fast. Meta’s Llama 4 and DeepSeek's range of models turns yesterday’s “good enough” hardware into today’s bottleneck, so picking the right platform is more strategic than...
Article
Explore NVIDIA's GTC 2025 reveals, from Hopper racks to Silicon Photonics. What challenges and surprises await digital infrastructure's future?
Article
Expanding on my post from last week, it was great to see AWS leaning back into their engineering roots at re:Invent this year. They really pulled back the curtain to show us how they’re solving problems at a scale most...
Article
For the final piece of the Global Summit wrap up, I focus on Networking, both inside the server and between racks, and the crystal ball state of AI. Networking In the area of networking, the conference shed light on some...
Article
We've touched on the power innovations at the summit, so obviously, the next logical step is to talk about cooling. Cooling If there is one thing I noticed this year, was the surge in companies expanding their portfolios...
Article
Originally planned as a two-part reflection, my series from the fantastic OCP Summit has grown into a series! Up next: the topic of power. Power One of the most talked about topics at the conference was power....
Article
It’s been a busy conference season, with the AI Hardware and Edge AI Summit, Yotta 2024, and OCP’s Global Summit all taking place in the past month or so. The OCP Global Summit has become a personal favorite of mine; the...
Article
As I’ve been busy with my day job(s) and various projects, like the Tech Insider Podcast, I haven’t put my hands to the keyboard for an article in a while. In this piece, while I’m still talking about AI, I want to...
Article
Nvidia's latest roadmap was teased at Computex in Taiwan last month. Whilst details were a little light on PFLOPS and TDP for either the GPU or CPU, we did get some interesting information for the next-gen products. GPU...
Article
Just last month, Apple hosted their yearly WWDC - an event where they showcase all the updates to their platforms. Whilst a lot of it is very interesting, and AI centric, I'm going to mostly focus on Private Cloud...
Article
As you may have seen (or heard my "Ausmerican" accent) recently, I've started a podcast, and wanted to share a little insight into why. I created this, "Infrastructure as a Newsletter", for * checks calendar * just shy...
Article
The Open Compute Project (OCP) Regional Summit was hosted in Lisbon, Portugal last month, the 5th (and largest) regional summit the group has hosted. Whilst I wasn't able to make it in person, I’d be remiss if I didn't...
Article
The tidal wave of Generative AI (GenAI) has mostly consisted of training large language models (LLM's), like GPT-4, and the huge amount of compute needed to process these enormous datasets, e.g. GPT-4 has 1.76 trillion...
Article
Upon returning from GTC24, I've been able to reflect on all the new updates across NVIDIA's platforms and below is a summary of the various announcements. Blackwell was the star of the show, with the B100, B200 and GB200...
Article
Next week, NVIDIA CEO Jensen Huang will take the stage at the SAP Center in San Jose to showcase the company's vision for the next year, and likely (see below) a few announcements too. Expectations are sky-high, like...
Article
In the evolving landscape of data center technology, the demand for efficient and sustainable cooling solutions continues to grow. With the emergence of GenAI infrastructure and its increased computational demands and...
Article
Happy New Year and welcome to 2024! After what felt like the 24th month of 2023 , I enjoyed some much-needed downtime with my family, and starting the new year refreshed where I expect 2024 to be a big year across the...
Article
If you are looking for the first half, check out my OCP 2023 Wrap (1 of 2) | LinkedIn For the second (and final) half, I'm concentrating on Immersion + Sustainability (which go hand-in-hand), and Disaggregation....
Article
As a follow up to my last article, the lovely folks at OCP published over 500 videos from the conference. FIVE HUNDRED. I didn't exactly plan for the sheer quantity of sessions that I would be interested in, and after...
Article
Posting about today's #microsoftignite2023 announcement in Seattle, from #sc23 in Denver, while finishing an article on #ocpsummit23 in San Jose is frankly...a little meta - but here we go. Whilst Microsoft cover a lot...
Article
Whether you're a startup seeking to lay a strong tech foundation, a mid-size company aiming to optimize infrastructure, or a large corporation lacking a comprehensive technology strategy, my expertise can help. With...
Article
Whether you're a startup seeking to lay a strong tech foundation, a mid-size company aiming to optimize infrastructure, or a large corporation lacking a comprehensive technology strategy, my expertise can help. With...
Article
Let's face it, technology isn't just a small piece of the business puzzle anymore; it's a potential game-changer that can make or break a company. Enter the dynamic duo of the tech world: the Chief Information Officer...
Article
In this article, we explore the world of immersion fluids used in cooling systems. Optimal performance, heat transfer efficiency, and system longevity hinge upon the selection of the most suitable immersion fluid....
Article
If you are Australian, and over 40, you might remember this as a nostalgic catchphrase from the 80's and 90's, advertising motor oil made by Castrol. While typically referred to as Oil, immersion cooling uses dielectric...
Article
In my last article, I discussed what a GPU was and primarily covered NVIDIAs history and product suite. While NVIDIA GPUs have long been the go-to choice for these AI workloads, there have been a host of alternative...
Article
The history of Graphics Processing Units (GPUs) is a fascinating journey that begins with their humble origins as tools for enhancing 3D graphics in video games. These early GPUs (then called 3D accelerators) such as the...
Article
As chips and designs continue to push boundaries with ever higher Thermal Power Densities (TPDs), managing the dissipated heat becomes increasingly challenging. Current air-cooled rack designs typically top out at around...
Article
Artificial Intelligence (AI) has become a pervasive buzzword in the industry, evoking various reactions including an amusing supercut of Sunar Pichai at Google's 2023 I/O... AI demands high-powered compute, which...
Article
In my previous article, I discussed the flexibility of immersion cooled infrastructure, particularly for edge deployments, as well as its potential to enable significant scalability for datacenter operators while...
Article
Immersion cooling offers a powerful solution for hosting high-value services such as AI and HPC workloads more cost-effectively in your datacenter. This approach leverages the superior cooling capabilities of immersion...
Article
In the ever-expanding digital landscape, data centers are the backbone of modern businesses, powering everything from cloud services to artificial intelligence applications. As datacenter infrastructure continues to...
Article
In the rapidly evolving world of datacenters, staying ahead of the curve is essential for businesses aiming to optimize performance, reduce costs, and minimize their environmental impact. One groundbreaking technology...
