Where Does a Vision Transformer Keep Its Sense of Geometry?

A self-supervised ViT can predict depth from a single image with no 3D supervision at all. That’s usually treated as one fact. A recent paper asks a sharper question: is that geometry sitting in a few clean directions the network can read out directly, or is it smeared across the whole feature space, waiting for a powerful decoder to untangle it?

Reading notes on Zhou et al., Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention · arXiv:2607.01987

Most evaluations of vision transformers stop at a single number: how well does a probe trained on top of frozen features perform? If it’s high, we say the model “understands” geometry. If it’s low, we shrug, maybe the information isn’t there, maybe it’s just hard to extract. This paper doesn’t accept that shrug. It opens the probe up and asks what it’s actually made of.

Daha fazla

Why AI Companies Are Building Their Own Chips (And What It Actually Buys Them)

A note on scope: chip architecture isn’t my research area. This is a synthesis of public reporting on a trend worth understanding, written with the same care about sourcing and overclaiming I try to apply to my own work.

In about 18 months, nearly every major AI lab and cloud provider has quietly become a chip company. Broadcom, now co-designing custom silicon with Google, Meta, and OpenAI, among others, is carrying a $73 billion AI backlog and telling investors it expects more than $100 billion in annual AI chip revenue by 2027. Amazon’s in-house silicon business, built to run in its own data centers rather than sold to others, is already generating revenue at a $20 billion annual run rate. Amazon, Google, Microsoft, and Meta combined are on track to spend roughly $725 billion on capital expenditure in 2026, a 77 percent jump from the year before, with a growing share of that aimed at infrastructure that isn’t a Nvidia GPU.

Daha fazla

Autoencoders: From Reconstruction to Representation Learning

1. Introduction

In the landscape of deep learning, models are typically designed to predict a target variable yy given an input 𝐱\mathbf{x}. Autoencoders, however, subvert this paradigm. At their core, an autoencoder is a neural network trained to reproduce its own input, effectively learning to approximate the identity function f(𝐱)≈𝐱f(\mathbf{x}) \approx \mathbf{x}.

While training a network to act as a simple “copy machine” might sound mathematically trivial, the true power of an autoencoder lies in its architectural constraints. By forcing the input data through a low-dimensional bottleneck before reconstructing it, the network is restricted from simply memorizing the input space. Instead, it is compelled to learn a compact, informative representation of the data’s underlying continuous manifold. This compressed latent representation serves as a powerful foundation for a multitude of advanced downstream tasks.

Daha fazla

From ANI to AGI: Understanding the Spectrum of Artificial Intelligence

The journey from Artificial Narrow Intelligence (ANI) to Artificial General Intelligence (AGI) represents one of the most ambitious and profound quests in the field of artificial intelligence. This voyage is not merely a technical endeavor but a journey towards realizing a dream that has captivated scientists, philosophers, and dreamers alike for decades. The aspiration to create an AI system that rivals human intelligence in its generality and versatility is both a source of inspiration and a monumental challenge. As we stand on the shoulders of today’s technological advancements, it’s crucial to understand the spectrum of artificial intelligence, distinguishing between the tangible achievements of ANI and the elusive horizon of AGI. This blog post aims to explore this distinction, shedding light on the current state of AI and the path that may lead us toward achieving true general intelligence.

Daha fazla

Understanding Neural Networks Through Demand Prediction

In the fast-paced world of retail, predicting which products will capture the market’s attention is more than just a guessing game; it’s a science. This is where the power of neural networks comes into play, transforming vast amounts of data into actionable insights. At the heart of this transformation is the ability to accurately predict demand, ensuring retailers can make informed decisions on inventory levels and marketing strategies. But what exactly are neural networks, and how do they manage to turn data into predictions?

Daha fazla