Q2 2023 IT Review – Generative AI, PaLM 2, Apple Vision Pro

AutoGPT is, by any measure, the viral phenomenon of Q2 2023. Released in late March and exploding in April, it accumulates over 100,000 GitHub stars in roughly two weeks — a pace no AI repository has reached before. In short, the project’s central proposition is disarmingly simple: give a GPT-4 instance a goal, a set of tools (web search, file I/O, code execution), and permission to plan multi-step tasks autonomously without user intervention at each step. In practice, the agent writes its own sub-tasks, executes them, evaluates the results, and iterates.

AutoGPT is compelling in April 2023, though not because it always succeeds. It frequently enters circular reasoning loops or gets lost in rabbit holes. What makes it compelling is that it makes autonomous AI agency feel tangible and near. For example, some engineers spend an afternoon watching AutoGPT work. It attempts to research a business idea, write a report, and email it to a Notion workspace. They come away fundamentally changed in their sense of what the next software paradigm looks like. Moreover, the project popularises concepts — tool use, memory, task decomposition — that rapidly become standard vocabulary in the LLM community.

However, AutoGPT also reveals, in public, the unsolved engineering problems of agentic AI: hallucinated tool outputs, context window exhaustion on long tasks. In addition, the compounding of errors across multi-step chains. As a result, these failure modes prompt a wave of follow-on research into better agent architectures, memory systems, and reliability guarantees. In this sense, AutoGPT is as important for what it breaks as for what it builds.

If AutoGPT defines the imagination of Q2 2023, vLLM defines its infrastructure reality. Released in June 2023 by researchers at UC Berkeley, vLLM introduces PagedAttention — an attention algorithm that manages the key-value cache for transformer inference using an approach borrowed directly from operating-system virtual memory management. The insight is straightforward but profound. In practice, pre-allocating a contiguous KV-cache block per request wastes significant GPU memory through fragmentation and over-allocation. Instead, PagedAttention maps the KV cache into fixed-size “pages”. Those pages can be allocated, freed, and shared non-contiguously, exactly as an OS manages RAM pages.

Overall, the practical outcome is dramatic. vLLM delivers 2–4× higher serving throughput than the previous state-of-the-art systems, FasterTransformer and Orca, at the same latency level. Moreover, the improvement grows with longer sequences, larger models, and more complex decoding algorithms such as beam search — precisely the scenarios that matter most for production deployments. As a result, this makes vLLM the go-to inference backend for any team serving open-weight models at scale during Q2 and Q3 2023.

In addition, the project provides a drop-in OpenAI-compatible REST API, which dramatically lowers the barrier to deploying open-source models as API-compatible replacements for ChatGPT or Claude. For example, teams running LLaMA, Falcon, or StarCoder behind vLLM can reuse existing client code with zero changes. By the time the formal PagedAttention paper appears at SOSP 2023 in October, the software has already become a production standard.

vLLM demonstrates something important about the maturity of the field: the performance gap between commercial and open-source LLM serving is closing rapidly, driven by principled systems engineering rather than raw model scale.

Meanwhile, Georgi Gerganov’s llama.cpp is a masterclass in low-level systems engineering applied to a high-level AI problem. The project starts with a simple goal: run Meta’s LLaMA models on commodity hardware. That means a MacBook, a home server, eventually a Raspberry Pi. In practice, the method is to convert the float32 model weights into 4-bit integers, a process called quantisation. In addition, the code is optimised inference written in plain C++. It exploits SIMD instructions and the Metal GPU API on Apple Silicon.

Notably, the timing is perfect. Meta’s LLaMA weights have leaked online in February 2023, and tens of thousands of developers want to experiment with them locally without cloud API costs or privacy concerns. In short, llama.cpp provides the runtime. Moreover, its GGML quantisation format — which produces compact model files in formats like Q4_0, Q4_1, Q5_K — becomes the de-facto standard for distributable open-weight LLMs throughout 2023 and beyond.

What makes llama.cpp so influential in Q2 2023 is the cascade of models it enables. For example, Alpaca, Vicuna, WizardLM, and dozens of other instruction-tuned LLaMA derivatives run cleanly through llama.cpp with minimal configuration. The project effectively democratises access to performant language models. A researcher in a country without access to OpenAI’s API can run a capable 7B-parameter model on their laptop. So can an engineer working with sensitive data they cannot send to a third-party server. The downstream effect on education, research access, and AI safety experimentation is substantial.

Hugging Face and ServiceNow release StarCoder in May 2023 as part of the BigCode project. It represents a principled answer to a question that haunts proprietary code-generation models. What is the provenance of the training data? Do the model weights respect the licenses of the code they are trained on? GitHub Copilot and Amazon CodeWhisperer are trained on vast corpora of public code. However, neither organisation publishes a detailed breakdown of which licenses are included or how attribution is handled.

BigCode takes a different approach. StarCoder is trained on The Stack, a curated dataset of permissively licensed source code (MIT, Apache, BSD) built with an opt-out mechanism so that individual developers can request their code be removed. The 15.5-billion-parameter model supports eighty-six programming languages and includes a special 8,192-token context window — far longer than contemporaries — that allows it to reason about multi-file codebases with meaningful inter-file context.

Free ebook

Free AI Video, Generated Locally

Working scripts and measured benchmarks. Free.

Your e-mail address

Leave this field empty

Get the ebook

No spam. Unsubscribe at any time.

On standard code benchmarks such as HumanEval, StarCoder performs well. It matches or beats much larger models from closed providers. That validates the hypothesis that data quality and curation matter at least as much as raw parameter count. More importantly, StarCoder is freely downloadable and commercially usable under the BigCode OpenRAIL license, making it an attractive choice for enterprises that need an auditable, self-hosted code assistant. The model’s release accelerates the enterprise adoption of open-source LLMs throughout the second half of 2023.

Meta AI’s Segment Anything Model (SAM), released in April 2023, does for computer vision segmentation what GPT-3 did for natural language generation: it turns a labour-intensive specialist task into a promptable, general-purpose API. SAM is trained on over one billion masks generated from 11 million images using a sophisticated data-engine pipeline, making it the largest segmentation dataset ever assembled. The model accepts points, bounding boxes, or free-form text as prompts and returns high-quality segmentation masks in real time.

The practical implications reach well beyond research. Medical imaging teams use SAM to annotate anatomical structures. Satellite imagery analysts apply it to land-use mapping. Video editors use it to isolate objects across frames without frame-by-frame manual work. The model’s zero-shot generalisation — meaning it works competently on object categories it has never seen during training — marks a genuine conceptual advance in the field.

SAM’s release under the Apache 2.0 license, combined with Meta’s commitment to open-weight research, reinforces the emerging open-source strategy at the company: accept short-term competitive disadvantage in exchange for ecosystem influence and talent magnetism. This open-weights philosophy, also embodied in the LLaMA release, positions Meta as the primary institutional counterweight to OpenAI’s closed-model approach and proves enormously influential throughout 2023.

Microsoft’s Guidance library, released in April 2023, addresses one of the most persistent pain points in LLM application development: the unpredictability of model outputs when a program needs structured data. Conventional prompt engineering asks a model to “return valid JSON” and then parses the result, hoping the model cooperates. Guidance takes a different approach. It interleaves generation and logic at the token level. That lets developers write templates that constrain generation in real time. Those templates enforce JSON schemas, select from enumerated values, and branch on generated content.

The library introduces a template language that combines Handlebars-style syntax with Python logic. Constructs look like “generate a list of exactly five items”, or “if the generated category is X, then generate a follow-up question about X”. Constraints are applied at the generation stage rather than in post-processing. Therefore the model cannot “escape” the schema. Either it generates a valid token or the library forces a valid alternative. The result is dramatically more reliable structured output.

Guidance is particularly influential in enterprise contexts where LLM outputs feed downstream systems: form parsers, database insertion pipelines, validation frameworks. Its approach is later adopted or influenced by a generation of structured-generation libraries, including Outlines, Instructor, and SGLang. In the broader context of Q2 2023, it represents the engineering community’s growing focus on reliability and predictability as production LLM use cases multiply.

Sam Altman occupies an unusual position in Q2 2023: he is simultaneously the person most responsible for accelerating AI deployment at scale and the most prominent voice calling for regulation of his own industry. Notably, his May 16 Senate testimony is a masterclass in political positioning. He acknowledges risks clearly enough to be credible on safety. At the same time, he frames proposals such as licensing regimes and a federal AI agency. By comparison, a large, well-resourced company like OpenAI is better positioned to comply with those than smaller competitors.

Whatever one’s view of the strategic dynamics, Altman’s testimony shifts the Overton window in Washington. Before May 2023, the dominant DC posture on AI is “wait and see.” After Altman’s appearance — and the bipartisan agreement it generates — the question is no longer whether to regulate but how. As a result, this creates a faster regulatory timeline than the AI industry had anticipated. In addition, forces legal and compliance teams at every major AI company to accelerate internal governance work.

In addition, Altman tours Europe in June 2023, meeting with EU commissioners and national government officials. His public statements during this tour at times suggest OpenAI might leave Europe. That would follow if the AI Act’s requirements prove too burdensome. He then quickly walks those comments back. Those statements illuminate the tension between global AI deployment ambitions and local regulatory authority. That tension defines the next several years of AI governance negotiation.

Dario Amodei co-founds Anthropic in 2021 after leaving OpenAI, motivated by the belief that AI safety research must be embedded in the organisations building the most capable models, not conducted at arm’s length in academic labs. Q2 2023 is a validation quarter for this thesis. For example, the Claude API launch carries a 100,000-token context window and Constitutional AI training methodology. Constitutional AI teaches the model to reason about its own outputs against a set of principles. The launch demonstrates that safety-focused design choices can coexist with capability, and potentially enhance it.

Constitutional AI, Anthropic’s training approach, uses AI feedback rather than exclusively human feedback to align model behaviour. The model is first prompted to critique its own outputs against a written constitution of principles, then trained on those critiques. The approach scales better than pure human labelling and produces models that are notably more consistent in declining harmful requests while remaining helpful for legitimate use cases.

Notably, Google’s $300M+ strategic investment in Anthropic, confirmed during Q2 2023, is significant for two reasons. First, it validates the safety-first framing at a commercial level — Anthropic is fundable on the basis of its safety credentials as much as its capabilities. Second, it signals that the compute and distribution advantages of a hyperscaler partnership are available to companies outside the OpenAI-Microsoft alliance, reshaping the competitive landscape for the remainder of the decade.

Andrej Karpathy returns to OpenAI in May 2023 — he had previously left to join Tesla as Senior Director of AI — and immediately becomes one of the most influential technical educators in the LLM space. His YouTube channel, where he delivers multi-hour lectures building neural networks and LLMs from mathematical first principles, accumulates millions of views. The “Let’s build GPT” video provides a generation of software engineers with an accessible but rigorous path from gradient descent to transformer attention.

Karpathy’s pedagogical approach — deriving everything from scratch, showing the code, testing hypotheses interactively — models the kind of deep understanding that many engineers working with LLMs in Q2 2023 feel they lack. In practice, there is a growing gap between people who can call the OpenAI API and people who understand what happens inside the model when they do. Overall, Karpathy’s work narrows that gap more effectively than almost any other resource in this period.

Beyond the videos, Karpathy popularises terminology — “vibe coding,” the “software 2.0” paradigm — that shapes how the engineering community thinks about the transition from hand-coded logic to learned models. His ability to translate cutting-edge research into accessible intuition makes him unusually valuable during a period when the research-to-deployment cycle has compressed from years to months.

Simon Willison — co-creator of Django and current maintainer of the Datasette project — is one of the most consistent, practical, and widely-read voices on LLMs throughout Q2 2023. For example, his blog publishes near-daily notes on new models, tools, and techniques, written from the perspective of an experienced developer experimenting seriously with each new capability. The entries are dense with actual code, actual outputs, and actual failures. That is a reliable antidote to the breathless press coverage that surrounds every model release.

Willison’s particular contribution to Q2 2023 is his systematic documentation of prompt injection attacks — a class of vulnerability in which adversarial text embedded in documents, web pages, or user inputs hijacks an LLM’s instruction-following behaviour. For instance, his writing on this topic includes detailed proof-of-concept demonstrations. It brings a real security concern to the attention of a much wider engineering audience. It also forces early-stage LLM application developers to think about input validation and sandboxing. They have not encountered those concerns before.

More broadly, Willison occupies a useful role as the “pragmatic practitioner” in a conversation dominated by either research abstractions or product marketing. In practice, his work reflects how LLMs actually behave when deployed in real applications. This is frequently different from both the optimistic capability claims of vendors and the catastrophist predictions of critics. Overall, his influence in shaping the technical culture around LLM tooling in Q2 2023 is significant.

Yann LeCun — Turing Award winner, inventor of convolutional neural networks, and Meta’s Chief AI Scientist — is the most prominent dissenter from the consensus AI-risk narrative that solidifies during Q2 2023. Against the backdrop of the extinction-risk statement and Geoffrey Hinton’s departure from Google, LeCun argues publicly and persistently that autoregressive language models are not on a path to general intelligence, that the doom warnings are overblown. In addition, he argues that the real risks of current AI are bias, misinformation, and corporate power concentration.

LeCun articulates an alternative research agenda in a position paper titled “A Path Towards Autonomous Machine Intelligence”. It proposes a new architecture for world models. That architecture is based on energy-based models, self-supervised learning, and hierarchical planning. It is a fundamentally different approach from the LLM paradigm. His architecture proposal may or may not prove correct. His conceptual argument is that the current generation of LLMs is missing critical components. Those components are needed for genuine understanding and agency. That argument is technically substantive. It provides a valuable counterweight to capability extrapolations from scaling curves.

LeCun is prolifically present on LinkedIn and Twitter in Q2 2023. There he engages directly and often combatively with critics. That makes him a focal point for debates about AI timelines and safety research priorities. It also makes him central to debates about the relationship between academic and industrial AI. His combination of technical authority, institutional position at a company actively releasing open-weight models, and rhetorical willingness to argue gives his voice an influence that shapes research agendas and funding conversations well beyond the quarter.