Part of IT Trends & Reviews — what actually shipped, quarter by quarter.
1. Introduction: Convergence at Scale
The third quarter of 2019 is, without exaggeration, one of the most technically consequential three-month windows in the preceding decade. In fact, several forces that have been building for years simultaneously crest in this quarter. Overall, the result is a period that reshapes every layer of the technology stack, from silicon to language models.
Notably, the dominant hardware story is AMD’s Zen 2 architecture, which materialises not as a modest improvement but as a generational leap. For example, on July 7, AMD launches the Ryzen 3000 desktop processors. On August 7, it follows with second-generation EPYC Rome server chips. Both are based on a 7-nanometre process node manufactured by TSMC. Notably, the top-of-the-range Ryzen 9 3900X offers 12 cores and 24 threads at a $499 launch price and matches or exceeds Intel’s Core i9-9900K in nearly every multi-threaded benchmark. In the server market, the impact is even larger. For example, a dual-socket EPYC Rome system delivers 128 cores and 256 threads. Its total cost of ownership is one that cloud providers and HPC operators simply cannot ignore. After roughly a decade of Intel setting server CPU prices with minimal competition, the arithmetic of the data centre changes this quarter.
1.1 Software, strategy, and the trade backdrop
Meanwhile, on the software front, Google’s TensorFlow team releases RC0 of TensorFlow 2.0 in July and, exactly 91 days later, ships the general availability release on September 30. TensorFlow 2.0 is not an incremental version bump. Rather, it is a ground-up redesign. It replaces the static-graph, session-based programming model with eager execution by default. It integrates Keras as the canonical high-level API. Finally, it ships a tf.function decorator that provides graph-level performance when you need it. Additionally, the Hugging Face library — previously known as pytorch-transformers — is renamed simply transformers in August 2019. It simultaneously adds TensorFlow 2.0 support. For the first time, a single library provides state-of-the-art pre-trained NLP models across both major ML frameworks. As a result, the practical implication is immediate: fine-tuning BERT, XLNet, or GPT-2 for downstream tasks becomes accessible to any Python developer.
Moreover, beyond silicon and software, this quarter carries significant strategic weight. For example, on July 22, Microsoft announces a $1 billion investment in OpenAI — a commitment that at this moment appears extraordinary given that GPT-2 is still the primary public product. However, that in retrospect represents one of the most prescient capital allocations in technology history. Meanwhile, Huawei responds to the US Entity List ban with the announcement of HarmonyOS in August. In addition, Cloudflare completes its initial public offering on September 13 at a $4.4 billion valuation. That signals that infrastructure and security software commands serious public market interest.
Meanwhile, the trade tensions between the United States and China continue to reverberate throughout the quarter. The semiconductor supply chain, the 5G standards debate, and the competitive positioning of Chinese technology companies all evolve under the same shadow. That shadow is tariffs and export controls, first imposed in Q2 2019 and now fully in force.
2. GitHub Deep Dive – Key Repositories of Q3 2019
Overall, the GitHub activity of Q3 2019 concentrates heavily around two themes: the maturation of the ML framework ecosystem as TensorFlow 2.0 approaches its final release. In addition, the emergence of a new category of libraries built on top of pre-trained transformer models. Moreover, Kubernetes continues its rapid evolution at the infrastructure layer. In practice, six repositories define the quarter’s open-source narrative.
2.1 TensorFlow 2.0 — tensorflow/tensorflow
TensorFlow 2.0 RC — A Framework Redesigned for Practitioners
TensorFlow 2.0 represents the most significant architectural change since TensorFlow 1.0 shipped in February 2017. Notably, the core change is the removal of the session-based, define-then-run programming model. By default, operations execute eagerly — meaning that a call to tf.matmul(a, b) immediately returns a tensor value, just as any NumPy operation would. This seemingly simple change eliminates one of the most common complaints about TensorFlow 1.x. That complaint was the disconnect between writing code and understanding what it does. Understanding required feeding dummy data through a tf.Session just to inspect intermediate values.
The @tf.function decorator bridges eager and graph execution. By decorating a Python function with @tf.function, TensorFlow traces the function’s operations and compiles them into a static computation graph. That enables XLA optimisations and distributed-training backends. It also preserves the readable Python code that developers write. The decorator is effectively a performance dial: write in eager mode for experimentation and debugging, add @tf.function when you need production throughput. Keras becomes the officially endorsed high-level API, replacing the confusing array of tf.layers, tf.estimator, and slim approaches from TF 1.x. Notably, at the time of the GA release, TensorFlow accumulates approximately 140,000 GitHub stars — comfortably the most-starred ML framework repository.
2.2 Hugging Face Transformers — huggingface/transformers
Transformers 2.0 — The NLP Swiss Army Knife
Moreover, the renaming of the Hugging Face library from pytorch-transformers to simply transformers in August 2019 is not merely cosmetic. In addition, the version 2.0 release adds TensorFlow 2.0 support. Models like BERT, GPT-2, XLNet, and RoBERTa are now accessible. That holds regardless of which framework a team has standardised on. By comparison, before this release, moving from a PyTorch pre-trained model to a TensorFlow deployment pipeline requires manual weight conversion. After it, a single line of Python loads the same checkpoint in either framework.
Furthermore, the library introduces a unified API — AutoTokenizer and AutoModel classes — that allows code to load any supported model without knowing its specific class at import time. Overall, this design foreshadows the model hub that Hugging Face will build in subsequent years. In Q3 2019, the transformers library has approximately 5,000 GitHub stars and is growing rapidly among NLP practitioners who want to apply BERT-style fine-tuning without rewriting Google’s research code.
2.3 PyTorch 1.2 — pytorch/pytorch
PyTorch 1.2 — TorchScript Matures and Quantisation Arrives
Meanwhile, PyTorch 1.2 ships on August 8 with improvements across three key areas. First, TorchScript is the JIT compiler that converts Python-defined PyTorch models into a representation that can run without a Python interpreter. It reaches a more complete state. It supports a wider subset of Python control flow. This directly addresses PyTorch’s production deployment gap relative to TensorFlow’s SavedModel format. Second, the release includes experimental support for quantisation-aware training and post-training static quantisation, reducing model size and inference latency at the cost of minimal accuracy degradation. Third, ONNX opset 9 export improves interoperability with production runtimes such as ONNX Runtime and TensorRT. Additionally, the C++ frontend API stabilises significantly. That enables PyTorch models to be embedded directly in C++ applications. It is a requirement for deployment in automotive and embedded systems that previously favoured TensorFlow.
2.4 Kubernetes 1.16 — kubernetes/kubernetes
Kubernetes 1.16 — Cleaning the API Surface
Kubernetes 1.16 is notable primarily for what it removes rather than what it adds. The release deprecates and removes several beta API versions that have existed alongside their stable counterparts for multiple release cycles: extensions/v1beta1 and apps/v1beta1 Deployments, StatefulSets, DaemonSets, and ReplicaSets are all removed, with apps/v1 now the only supported version for these resources. For teams running production clusters, this is a meaningful operational event — Helm chart deployments and CI/CD pipelines that target deprecated API versions break on upgrade. Furthermore, the release promotes Custom Resource Definitions (CRDs) to GA under apiextensions.k8s.io/v1, a graduation that signals Kubernetes’ operator pattern is now considered stable and production-ready. The operator ecosystem — which allows application-specific operational knowledge to be encoded in Kubernetes controllers — begins accelerating rapidly after this release.
2.5 Rust 1.36 and 1.37 — rust-lang/rust
Rust 1.36 and 1.37 — Async on the Horizon
Rust ships two releases in Q3 2019, both building anticipation toward the async/await syntax that is landing in Rust 1.39 (November 2019). For example, Rust 1.36 stabilises the Future trait — the foundational primitive for asynchronous Rust — and ships MaybeUninit, a safer alternative to mem::uninitialized(). Rust 1.37 adds cargo vendor for offline builds. That allows projects to vendor all dependencies into a local directory. The feature matters for security-conscious enterprises and air-gapped production environments. The Rust community grows substantially in Q3 2019, with surveys indicating that adoption in systems programming, WebAssembly targets, and embedded development is accelerating. Mozilla, Amazon, and Microsoft all actively contribute to Rust tooling during this period.
2.6 Megatron-LM — NVIDIA Research
Megatron-LM — Large-Scale Transformer Training at GPU Scale
NVIDIA’s Applied Deep Learning Research team open-sources Megatron-LM in September 2019. A research paper accompanies it. The paper describes techniques for training transformer models at scales that a single GPU cannot hold in memory. The team terms the approach model parallelism. It partitions the transformer’s attention layers and feedforward networks across multiple GPUs. That enables training of an 8.3-billion-parameter language model on 512 NVIDIA V100 GPUs. This is roughly five times the size of the largest GPT-2 model that OpenAI has publicly released at this point. For researchers and organisations tracking the trajectory of language model scaling, Megatron-LM is a harbinger. It demonstrates that model size is still a tractable scaling dimension. It also shows that larger GPU clusters directly translate to more capable language models.
3. Big Tech & Industry Breakthroughs
Q3 2019 delivers a concentrated sequence of major industry events spanning hardware launches, strategic investments, and IPOs across the Americas, Europe, and Asia. The following table provides an overview before each storyline is examined in depth.
| Company | Event | Date | Significance |
|---|---|---|---|
| AMD | Ryzen 3000 (Zen 2) desktop launch | Jul 7, 2019 | 7nm CPUs match Intel at half the price; CPU competition restarts at the high end |
| AMD | EPYC Rome (2nd Gen) server launch | Aug 7, 2019 | Up to 64 cores per socket; disrupts Intel Xeon’s decade-long data centre dominance |
| Microsoft | $1 billion investment in OpenAI | Jul 22, 2019 | Largest single AI investment of 2019; establishes Azure as OpenAI’s exclusive cloud provider |
| TensorFlow 2.0 GA | Sep 30, 2019 | Eager execution default; Keras unified API; resets ML framework developer experience | |
| NVIDIA | GeForce RTX Super series launch | Jul 2–23, 2019 | RTX 2060/2070/2080 Super deliver better price-to-performance for ray tracing and CUDA compute |
| Intel | 10th Gen Ice Lake laptop CPUs (10nm) | Jul 2019 | Intel’s first 10nm consumer CPUs; integrated Iris Plus GPU substantially improves laptop graphics |
| Apple | iPhone 11 / 11 Pro announcement | Sep 10, 2019 | A13 Bionic chip; triple-camera Pro system; computational photography leads mobile AI |
| Samsung | Galaxy Note 10 / 10+ announcement | Aug 7, 2019 | Exynos 9825 / Snapdragon 855; 5G variant available; S-Pen gains Bluetooth gestures |
| Huawei | HarmonyOS announced; Kirin 990 5G revealed | Aug / Sep 2019 | Response to Android access restrictions; Kirin 990 5G is the world’s first 5G SoC built on 7nm EUV |
| Cloudflare | IPO on NYSE (ticker: NET) | Sep 13, 2019 | Opens at $18; market cap approximately $4.4B; validates internet infrastructure as a public-market story |
3.1 AMD — Zen 2 Changes the Economics of Compute
AMD’s July 7 launch of Ryzen 3000 desktop processors is arguably the most consequential hardware event of the year. The Zen 2 microarchitecture combines with TSMC’s 7nm process node. It delivers approximately 15% improvement in instructions-per-clock (IPC) over Zen+. At the same time it doubles the maximum core count from 8 to 12 in the consumer segment. The Ryzen 9 3900X, at $499, demonstrates benchmark scores at or above Intel’s Core i9-9900K. That chip retails for approximately $500. However, the 3900X offers 12 cores versus Intel’s 8. Multi-threaded workloads including Blender rendering, video encoding, scientific computing, and deep learning data preprocessing see disproportionate gains.
However, the server announcement one month later carries even greater long-term weight. AMD announces its second-generation EPYC “Rome” processors on August 7, and they ship immediately to major cloud providers. They offer configurations of up to 64 cores per socket using a chiplet design. That design pairs small compute dies manufactured at 7nm with a larger I/O die at 14nm. A dual-socket Rome server delivers 128 physical cores and 256 threads, with memory bandwidth of approximately 3.4 TB/s using eight channels of DDR4 per socket. Google, Microsoft Azure, Oracle Cloud, and several European cloud providers all announce EPYC Rome-based instances during Q3 2019. For workloads that benefit from high core counts — containerised microservices, in-memory databases, and parallel ML inference — the price-per-core advantage over Intel Xeon Scalable processors is significant. Intel’s response, the Cooper Lake and Ice Lake Xeon families, remains months away.
3.2 The Radeon refresh and the 7nm GPU line
Additionally, AMD refreshes its GPU portfolio with Radeon RX 5700 and RX 5700 XT on July 7. Both are based on the new RDNA (Radeon DNA) architecture. They are also manufactured at TSMC 7nm. The RX 5700 XT competes with NVIDIA’s RTX 2070 at a lower price, representing AMD’s first competitive high-end GPU in several years. While the cards do not support hardware ray tracing, the rasterisation performance and compute throughput are both competitive — a material change from the Vega architecture generation.
3.3 Microsoft — The OpenAI Bet
On July 22, Microsoft announces a $1 billion strategic investment in OpenAI, framed as a multi-year partnership intended to accelerate OpenAI’s research into artificial general intelligence. As part of the arrangement, OpenAI commits to using Microsoft Azure as its exclusive cloud provider for training and deployment workloads. From Microsoft’s perspective, the deal is straightforward. OpenAI’s research trajectory at this moment includes GPT-2 and reinforcement learning work. It represents future compute demand that runs on Azure hardware. From OpenAI’s perspective, the investment provides capital to continue scaling experiments without near-term pressure to generate revenue.
At this point in Q3 2019, the magnitude of the bet is not obvious to outside observers. GPT-2 is a capable language model. However, it is primarily known for generating plausible news articles and for OpenAI’s unusual decision to withhold the full 1.5-billion-parameter model from public release in February 2019. The term “large language model” has not yet entered mainstream technology discourse. Nevertheless, the structural logic of the Microsoft-OpenAI partnership — compute provider and research organisation — proves prescient for the industry dynamics that unfold in subsequent years.
Free ebook
Free AI Video, Generated Locally
Working scripts and measured benchmarks. Free.
No spam. Unsubscribe at any time.
3.4 Google — TF 2.0 and the Leaked Quantum Supremacy Paper
Google’s Q3 2019 narrative has two distinct threads. The first is TensorFlow 2.0, which represents three years of community feedback translated into framework redesign. The RC process runs from RC0 in July through RC1 on August 29 to GA on September 30. That is unusually extended for Google software. It reflects the scale of the migration challenge. The installed base of TensorFlow 1.x code is enormous. It spans research, enterprise MLOps pipelines, and production serving infrastructure. In addition, the 1.x-to-2.x migration is not trivially backward-compatible.
The second thread is more dramatic. In late September, the Financial Times reports on a draft research paper. It describes Google’s achievement of “quantum supremacy”. That term means a demonstration that a quantum computer performs a specific task faster than any classical supercomputer. The paper was briefly published on a NASA website before being taken down. Notably, the paper claims that Google’s Sycamore quantum processor completes a specific sampling problem in 200 seconds. The same problem would require approximately 10,000 years on Summit. Summit is then the world’s most powerful classical supercomputer. IBM immediately contests the 10,000-year estimate, arguing that a classical algorithm on Summit could accomplish the task in roughly 2.5 days. Finally, the official announcement is staged for Q4. However, the leak triggers substantial discussion in Q3 2019 about where quantum computing stands and what “supremacy” actually means for practical applications.
3.5 Huawei — HarmonyOS and the 5G SoC Race
Meanwhile, Huawei holds its developer conference in August 2019 and announces HarmonyOS. It was initially called Hongmeng OS in Chinese communications. It is a distributed operating system designed to run across smartphones, smart speakers, tablets, automotive displays, and IoT devices. The announcement is positioned as a contingency plan and future direction rather than an immediate Android replacement. Huawei continues shipping Android on its phones. The Google Services restrictions come from the May 2019 Entity List addition. They apply only to future devices not already in the pipeline. However, HarmonyOS signals a strategic intent to reduce dependency on US software platforms. In addition, the developer ecosystem implications are significant for the Chinese technology market.
In addition, at IFA Berlin in September 2019, Huawei reveals the Kirin 990 5G chip. It is a system-on-chip manufactured on TSMC’s 7nm EUV (extreme ultraviolet lithography) process. It has an integrated 5G modem. The Kirin 990 5G is the world’s first commercial 5G SoC manufactured at 7nm EUV. That gives Huawei a temporary technical lead over Qualcomm’s Snapdragon 865. By contrast, the Snapdragon separates its 5G modem onto a discrete chip and ships several months later. In practice, the Kirin 990 5G powers the Mate 30 series announced on September 19. Despite strong hardware specifications, the Mate 30 series ships without Google Mobile Services. It is the first flagship Huawei device affected by the Entity List restrictions. That limits its commercial appeal outside China.
3.6 Apple — iPhone 11 and A13 Bionic
Meanwhile, Apple’s September 10 event at Steve Jobs Theater introduces the iPhone 11 and iPhone 11 Pro family, powered by the A13 Bionic chip manufactured by TSMC at 7nm. Notably, the A13 is Apple’s most powerful mobile processor to date, delivering approximately 20% faster CPU and GPU performance than the A12. However, the defining feature of the A13 is its updated Neural Engine — an 8-core, 6-TOPS (trillion operations per second) accelerator dedicated to on-device machine learning inference. The Neural Engine powers the iPhone 11’s computational photography features. These include Deep Fusion and Night Mode. Deep Fusion applies per-pixel machine learning processing to optimise texture and detail in non-HDR shots. For example, the A13’s Neural Engine processes roughly 1 trillion operations per photo in certain modes, illustrating the practical scale of on-device ML at this stage.
4. AI & Technology Impact
Overall, for Q3 2019, the AI landscape is firmly in the era of transfer learning and large pre-trained language models. The academic and industry communities are absorbing BERT — released by Google in November 2018 — and exploring its limits. Crucially, this quarter witnesses the emergence of the next wave of NLP models and the first serious industrial-scale training efforts that hint at where model size is heading.
4.1 Q3 2019 AI and Technology Timeline
TensorFlow 2.0 RC0 Released
Google releases the first release candidate of TensorFlow 2.0, triggering a two-month community evaluation period. The RC includes eager execution, the @tf.function decorator, unified Keras API, and updated documentation. As a result, enterprise adoption teams begin migration testing.
AMD Launches Ryzen 3000 and Radeon RX 5700 Series
Notably, AMD’s double launch on a single day sends a clear signal that the company is fully competitive across CPU and GPU segments. For example, the Ryzen 9 3900X at $499 offers 12 cores on 7nm. Similarly, RX 5700 XT challenges RTX 2070 at a lower price, based on the new RDNA architecture.
Microsoft Announces $1 Billion Investment in OpenAI
The investment is structured as a partnership: OpenAI will use Azure exclusively for compute, and Microsoft gains preferred commercial rights to productise OpenAI technologies. Overall, the deal values OpenAI’s research agenda and makes Azure a foundational piece of the most ambitious AI safety and capabilities research programme in the world.
AMD EPYC Rome Server CPUs Launch; Samsung Galaxy Note 10 Announced
In addition, EPYC Rome delivers up to 64 cores per socket on 7nm. As a result, major cloud providers immediately announce instance types based on the new chips. On the same day, Samsung reveals Galaxy Note 10 and Note 10+, with 5G variants available in select markets from August 23.
PyTorch 1.2 Released
Meanwhile, Facebook AI releases PyTorch 1.2 with improved TorchScript, experimental quantisation support, and better ONNX export. As a result, the release narrows the deployment gap between PyTorch and TensorFlow and strengthens PyTorch’s position in production serving pipelines.
Hugging Face Renames Library to “Transformers”, Adds TF 2.0 Support
The renaming from pytorch-transformers to transformers signals the library’s ambition to become the standard NLP toolkit regardless of framework. Moreover, adding TensorFlow 2.0 support alongside the existing PyTorch backend immediately increases the library’s total addressable developer audience. In practice, BERT, GPT-2, XLNet, RoBERTa, and DistilBERT are all available through a unified API.
4.2 Q3 2019 AI and technology timeline: late August to September
OpenAI Releases GPT-2 Large (774M parameters)
OpenAI publishes the 774M-parameter “large” version of GPT-2, continuing its staged release strategy. The small (117M) model was released in February; medium (345M) in May. By comparison, the large model demonstrates substantially improved text coherence and factual recall. The full 1.5B model remains unreleased, pending OpenAI’s assessment of misuse potential.
NVIDIA Releases Megatron-LM; Trains 8.3B Parameter Language Model
NVIDIA’s Applied Deep Learning Research publishes Megatron-LM and a paper showing training of an 8.3-billion-parameter language model on 512 V100 GPUs. The model parallelism technique partitions transformer layers across GPU memory boundaries, enabling models that exceed single-GPU memory capacity by an arbitrary factor. The result is a blueprint for how to scale transformers well beyond GPT-2 scale.
4.3 Q3 2019 AI and technology timeline: mid-September onward
Cloudflare IPO on NYSE
Cloudflare prices its IPO at $15 per share, opens at $18, and closes its first day at $18.02 — a 20% first-day gain. The company’s market capitalisation reaches approximately $4.4 billion. Cloudflare’s unique position as a security and networking company serving tens of millions of websites generates substantial investor interest as a proxy for internet infrastructure growth.
Kubernetes 1.16 Removes Deprecated Beta APIs
The release is a watershed moment for cluster operations teams: Helm charts and YAML manifests targeting deprecated extensions/v1beta1 API versions fail to apply on v1.16 clusters. The Kubernetes community has been signalling these removals for multiple release cycles. However, the enforcement triggers urgent update work across the ecosystem.
Huawei Mate 30 Announced; Google Quantum Supremacy Leak
Huawei’s Mate 30 series, powered by Kirin 990 5G, is announced on September 19 without Google Mobile Services. The following day, the Financial Times reports on a Google research paper. It was briefly visible on a NASA collaboration site. It claims the Sycamore quantum processor has achieved quantum supremacy. IBM immediately challenges the classical baseline comparison.
TensorFlow 2.0 GA — End of Q3
Google ships TensorFlow 2.0 final release on the last day of Q3. The release includes eager execution by default, tf.function for graph-level performance, Keras as the canonical high-level API, and TF Serving 2.0 for production deployment. In addition, the migration guide and tf_upgrade_v2 tool address the significant backward-compatibility challenges for TF 1.x users.
4.4 The NLP Arms Race: From BERT to XLNet and RoBERTa
The NLP community in Q3 2019 is digesting a rapid succession of models that build on the transformer architecture introduced in “Attention Is All You Need” (Vaswani et al., 2017). BERT, published by Google in November 2018, establishes masked language modelling and next-sentence prediction as a pre-training objective that yields transferable representations for downstream tasks. However, in June 2019, just before Q3 begins, researchers from CMU and Google Brain publish XLNet. It addresses what they identify as a fundamental limitation in BERT’s training procedure. BERT’s masked tokens are independent of each other during training. By contrast, a language model benefits from understanding the dependency between masked positions.
XLNet uses a permutation-based language modelling objective. It randomly permutes the order in which tokens attend to each other. That captures bidirectional context while avoiding the training-inference discrepancy caused by mask tokens. Additionally, XLNet incorporates the Transformer-XL architecture, which extends the context window beyond the standard 512 tokens by caching hidden states from previous segments. On release, XLNet achieves state-of-the-art results on 20 NLP benchmarks including GLUE, SQuAD, and several sentiment analysis tasks, improving over BERT substantially on several. The Q3 2019 community debate centres on which pre-training strategy is fundamentally superior. Facebook AI releases RoBERTa in Q3 2019 as well. It makes the case that BERT’s performance is primarily limited by insufficient training data and time. Architectural choices matter less.
Furthermore, the Hugging Face transformers library ensures that practitioners have immediate access to all three models — BERT, XLNet, and RoBERTa — through a consistent fine-tuning interface. Notably, the gap between state-of-the-art research and practical application is narrowing dramatically. A practitioner with access to a single GPU can now fine-tune a pre-trained XLNet model. The task is domain-specific text classification, and it takes hours.
4.5 Hardware Compute Trends: The 7nm Era Begins
Q3 2019 is the quarter in which TSMC’s 7nm process node goes from a promising technology to the dominant manufacturing node for high-performance chips. AMD Zen 2 CPUs and GPUs, Apple A13 Bionic, Huawei Kirin 990 5G, and NVIDIA’s next-generation GPU (in development) all use 7nm or 7nm EUV. Intel, by contrast, is still shipping at 14nm++ for its mainstream desktop and server processors, having encountered severe manufacturing difficulties at its own 10nm node.
The competitive situation creates a structurally unusual period. Intel’s manufacturing process lags behind the leading edge by approximately two process generations. By contrast, AMD outright ships products on the best available manufacturing technology. Analysts estimate that TSMC’s 7nm delivers approximately 35–40% power reduction and 25–30% die-size reduction. The comparison is with 10nm Samsung/Intel-class processes. That enables AMD to pack more cores per die at lower thermals. It is exactly the advantage AMD exploits with 64-core EPYC Rome.
5. Key Voices & Thought Leaders
Several engineers, researchers, and writers shape how the technical community understands Q3 2019 through their published work, talks, and public communications. The five individuals below exert particular influence during this period.
Jeremy Howard — fast.ai Founder, Deep Learning Educator
Jeremy Howard continues to be one of the most influential voices in practical deep learning throughout Q3 2019. The fast.ai v1 library, built on PyTorch. In addition, the accompanying Practical Deep Learning for Coders course make state-of-the-art techniques accessible to practitioners without PhD-level theoretical backgrounds. In 2018, Howard and Sebastian Ruder co-authored ULMFiT, or Universal Language Model Fine-Tuning. It demonstrated that pre-trained language models could be effectively fine-tuned for classification tasks. Limited labelled data was enough. This work predates BERT and contributes to the foundations of the transfer learning paradigm that dominates NLP in Q3 2019. Howard is also vocal during this quarter about the risks of AI hype and advocates publicly for responsible AI deployment. His public commentary on Twitter and the fast.ai forums provides one of the clearest practitioner-oriented perspectives on the evolving landscape of NLP models.
5.1 Framework authors and researchers
François Chollet — Keras Creator, Google Brain Engineer
In Q3 2019, François Chollet is simultaneously finalising TensorFlow 2.0, where Keras is now the canonical high-level API. He is also working on one of the most provocative papers of 2019. “On the Measure of Intelligence” is published in November 2019, but it is clearly in development during Q3. Chollet argues that the field’s dominant measures of AI progress are inadequate proxies for general intelligence. Those measures are benchmark performance on specific tasks. He proposes a formal definition of intelligence as skill-acquisition efficiency. His Twitter output during Q3 2019 is full of commentary on the limits of NLP benchmarking. He also writes on the distinction between task-specific performance and genuine generalisation, and on the engineering philosophy behind TF 2.0’s API design. These perspectives resonate strongly with practitioners who observe that fine-tuned BERT models perform exceptionally on held-out test sets but can fail in unexpected ways on real-world inputs.
5.2 Practitioner voices from industry
Andrej Karpathy — Director of AI at Tesla
At Tesla’s Autonomy Day in April 2019, Karpathy presented the neural network architecture behind Tesla’s Full Self-Driving computer. He also presented its data pipeline. In Q3 2019 he refines that work. He also discusses publicly the operational realities of deploying neural networks at consumer-scale production. Tesla’s AutoPilot system processes camera data from hundreds of thousands of active vehicles. That creates a dataset for training lane detection, object recognition, and depth estimation. No academic lab can replicate it. Karpathy’s observations about the divergence between academic computer vision benchmarks and real-world deployment challenges carry significant weight. His point that models trained on curated ImageNet-derived datasets do not automatically generalise to the diverse conditions encountered by production self-driving systems is a consistent theme. His public writing and conference presentations make him one of the most respected practitioners in applied computer vision and autonomous systems.
Martin Kleppmann — Distributed Systems Author
Martin Kleppmann’s “Designing Data-Intensive Applications,” published in 2017, reaches peak cultural relevance in 2019. The book — universally referenced as “DDIA” in engineering communities — provides a rigorous treatment of distributed data systems. It covers replication, partitioning, transactions, and stream processing, along with the consistency guarantees that underpin them. In Q3 2019, as cloud-native architectures increasingly involve multiple stateful services communicating through event streams, DDIA becomes required reading for senior engineers and staff engineers in many organisations. Kleppmann continues writing and speaking on related topics, including collaboration algorithms (CRDTs) and the decentralisation of data management. His work provides the theoretical grounding that practitioners need. They reason about the Kubernetes operator pattern, distributed ML training, and microservice data consistency. All of those are actively debated in Q3 2019.
5.3 Kernel and systems voices
Linus Torvalds — Linux Kernel Maintainer
Linux kernel development continues at its steady cadence in Q3 2019, with Linux 5.2 released on July 7 (the same day as AMD’s Zen 2 launch). For example, Linux 5.2 includes improved support for Zstd compression in the kernel, a new mount API, and improvements to the Btrfs filesystem. Linux 5.3 is released at the end of September 2019. Torvalds returned to kernel maintenance in October 2018 following his brief step back. He is actively visible on the Linux Kernel Mailing List throughout Q3 2019. In practice, he reviews patches and debates API design decisions. He also maintains the collaborative but exacting review culture that has sustained the kernel project for nearly three decades. Containerisation makes Linux kernel capabilities increasingly relevant to every data centre operator. Torvalds’ work covers kernel performance, scheduler improvements, and hardware support. It directly influences the cloud-native workloads that define the enterprise technology landscape.
6. Trend Synthesis
Stepping back from the individual events of Q3 2019 and examining what they collectively signal reveals four converging themes that will define technology development for the following several years.
6.1 The Manufacturing Process Node Gap Creates New Winners
The most structurally significant development of Q3 2019 is not a software release or an AI research paper. It is AMD’s success in bringing 7nm chips to market ahead of Intel’s comparable node transition. AMD ships 7nm consumer chips first, then 7nm server chips. This matters because process node leads translate directly into performance-per-watt advantages that compound across every system level. A 7nm chip runs cooler and consumes less power. If thermal budget is not a constraint, it can instead run at higher clock speeds. It can also accommodate more functional units than an equivalent 14nm design.
Therefore, the transition to 7nm manufacturing in Q3 2019 foreshadows a period in which semiconductor leadership becomes intertwined with foundry partnerships. AMD’s partnership with TSMC, Apple’s exclusive TSMC relationship, and Huawei’s Kirin 990 on TSMC EUV all reflect a structural shift in how competitive advantage in chip design is obtained. Intel’s vertical model — designing and manufacturing its own chips on proprietary process nodes — faces mounting pressure as TSMC accelerates ahead. The implications extend beyond desktop and server CPUs. They reach ML accelerators, mobile processors, and the future silicon landscape for AI inference hardware.
6.2 The ML Framework War Converges on Usability
TensorFlow 2.0’s arrival demonstrates that the static computation graph model, while theoretically elegant, has lost the developer experience battle to PyTorch’s eager-by-default, dynamic approach. Google’s decision to make TF 2.0 eager-first is an acknowledgement of this. Simultaneously, PyTorch 1.2 invests in TorchScript and production deployment, acknowledging that eager execution alone is insufficient for industrial inference pipelines that require serialisation, portability, and optimisation. The two frameworks are converging toward the same target: flexible development environments with production-grade deployment stories.
Furthermore, the Hugging Face transformers library’s framework-agnostic pivot in August 2019 signals a new architectural layer above the frameworks themselves. As pre-trained models become standard starting points for NLP tasks, a model-centric library that abstracts over framework details offers more immediate productivity than the frameworks themselves. This layer covers model hubs, pre-trained weights, and tokenisation utilities. It becomes the primary interface through which most NLP practitioners interact with deep learning. That holds regardless of whether TensorFlow or PyTorch powers the computation underneath.
6.3 Microsoft’s Bet on AI as Infrastructure
The $1 billion OpenAI investment is not Microsoft’s first AI acquisition or partnership. However, it is structurally different from previous moves. Rather than acquiring an existing product with a customer base, Microsoft is investing in open-ended research with no guaranteed commercial output. The logic is that if AI capabilities continue to improve along the trajectory suggested by GPT-2, XLNet. In addition, the scaling results demonstrated by Megatron-LM, then the compute infrastructure required to train and serve these models represents a sustainable and growing revenue stream. Azure positions itself as the cloud provider for AI research organisations, beginning with OpenAI. That is a deliberate attempt to establish category leadership. The leadership translates into customer acquisition as AI moves from research to enterprise deployment.
The Megatron-LM result from NVIDIA, arriving in the same quarter, provides strong evidence that the scaling trajectory is indeed tractable. Training an 8.3-billion-parameter model in September 2019 is an extraordinary demonstration of what GPU clusters, model parallelism techniques, and sufficient capital can achieve. The community watching these numbers understands, even if not explicitly stating it, that the logical question is: what happens at 10x, 100x, or 1000x scale?
6.4 Geopolitical Friction Reshapes the Technology Supply Chain
Meanwhile, the Huawei Entity List restrictions, which took effect in Q2 2019, produce their first major commercial consequences in Q3 2019 with the Mate 30 series shipping without Google Mobile Services. The announcement of HarmonyOS is a direct response, and while HarmonyOS is not yet a mature platform, the intent is transparent: Huawei is investing in software infrastructure independence. TSMC manufactures the Kirin 990 5G SoC before any potential foundry restrictions take effect. It represents the last generation of Huawei flagship silicon built on leading-edge Western process equipment. That fact becomes more significant in retrospect.
Additionally, the ongoing US-China trade negotiations cast a shadow over semiconductor supply chains. They also weigh on 5G equipment procurement decisions across Europe and on investment flows between Chinese and American technology companies. For practitioners making technology choices in Q3 2019, the geopolitical dimension is still primarily a background factor. However, infrastructure decisions made in this quarter begin to manifest as consequential long-term choices. That is especially true around 5G network vendors, as the following year’s events develop.
Key Insight — Transfer Learning Turns NLP into a Library Import
The significance of a shared model hub is that it changes who can ship natural language features. Before, competitive NLP required a research team and a training budget. With pre-trained transformers and a common interface, it requires a dependency and a fine-tuning dataset. The strategic consequence for this quarter is that model quality stops being the differentiator for most products, and data access plus deployment cost become the places where advantage actually accrues.
7. Summary
Q3 2019 is a quarter defined by convergence — hardware, software, and capital alignment that creates conditions for accelerated change. AMD’s Zen 2 architecture delivers proof that process node leadership translates directly into market disruption. Ryzen 3000 launches on July 7 and EPYC Rome on August 7. Those dual launches restore competitive pressure to both the consumer CPU market and the data centre. The immediate implications reach cloud pricing, workstation economics, and the cost structure of training large ML models.
On the software side, TensorFlow 2.0 arcs from RC0 in July to GA on September 30. It provides the ML community with a framework that finally prioritises developer experience. It does so without sacrificing production capability. Simultaneously, Hugging Face’s transformers library reaches a milestone that makes state-of-the-art pre-trained NLP models accessible across both TensorFlow and PyTorch with a consistent API. The combination lowers the barrier to applying BERT-scale models to real tasks. Those tasks include text classification, question answering, and summarisation. The barrier drops to a single pip install and a few dozen lines of Python.
Strategic capital flows are equally notable. Microsoft’s $1 billion OpenAI investment, announced July 22, establishes a structural link between the world’s most ambitious AI safety and capabilities research lab and Microsoft’s Azure cloud infrastructure. The deal’s long-term significance is not apparent in Q3 2019. However, the architecture of the partnership — exclusivity, commercial productisation rights, compute provision — reflects a sophisticated reading of how AI transitions from research artifact to commercial product. Cloudflare’s September IPO demonstrates that infrastructure software — particularly internet networking and security — commands public market valuations that reflect its strategic importance, even when profitability remains years away.
7.1 Open questions and what practitioners should do
The open questions as Q3 2019 closes are substantial. The TF 2.0 migration from TF 1.x involves significant friction for large enterprise codebases. How quickly the installed base will actually migrate is unclear. AMD’s EPYC Rome advantage is real, but Intel’s response — Cooper Lake, Ice Lake Xeon, and eventually Sapphire Rapids — is already in development and will partially close the gap. The NLP models advancing through transformer architectures are improving impressively. However, whether scale continues to produce qualitative improvements — or hits a ceiling — is the central research question. And the leaked Google quantum supremacy paper raises a question that will not be definitively answered for many years. What does practical quantum advantage actually require beyond a carefully constructed sampling demonstration?
For practitioners, Q3 2019 is a quarter that demands attention and adaptation. The ML framework they standardised on is changing. Similarly, the hardware they deploy on is changing. The NLP toolkit they evaluate against research literature is changing. And the strategic relationships between the largest technology companies are shifting in ways that will influence platform choices for the next decade. It is, in short, a quarter that cannot safely be ignored.
8. Sources
https://blog.tensorflow.org/2019/09/tensorflow-20-is-now-available.html — TensorFlow Blog: “TensorFlow 2.0 is now available”, September 30, 2019
https://pytorch.org/blog/pytorch-1-2-and-domain-api-release/ — PyTorch Blog: “PyTorch 1.2 Released”, August 8, 2019
https://github.com/huggingface/transformers/releases/tag/v2.0.0 — Hugging Face Blog: Transformers v2.0 announcement, August 2019
https://kubernetes.io/blog/2019/09/18/kubernetes-1-16-release-announcement/ — Kubernetes Blog: “Kubernetes 1.16 Release Announcement”, September 18, 2019
https://www.amd.com/en/press-releases/2019-07-07-amd-smashes-the-12-core-barrier — AMD Press Release: Ryzen 9 3900X and Ryzen 3000 series, July 7, 2019
https://www.amd.com/en/press-releases/2019-08-07-amd-expands-epyc-rome-product-family — AMD Press Release: EPYC Rome, 2nd generation, August 7, 2019
https://blogs.microsoft.com/blog/2019/07/22/microsoft-invests-in-and-partners-with-openai — Microsoft Blog: “$1 Billion Investment in OpenAI”, July 22, 2019
https://openai.com/research/gpt-2 — OpenAI: GPT-2 research page; 774M model release August 20, 2019
https://arxiv.org/abs/1909.08053 — Shoeybi et al., “Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism”, arXiv September 2019
https://arxiv.org/abs/1906.08237 — Yang et al., “XLNet: Generalized Autoregressive Pretraining for Language Understanding”, arXiv June 2019
https://arxiv.org/abs/1907.11692 — Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach”, arXiv July 2019
https://www.cloudflare.com/press-releases/2019/cloudflare-announces-pricing-of-initial-public-offering/ — Cloudflare Blog: IPO announcement, September 2019
https://web.archive.org/web/20190821002901/https://consumer.huawei.com/en/press/news/2019/huawei-developer-congress-2019/ — Huawei: HarmonyOS announcement at Huawei Developer Conference, August 2019
https://www.apple.com/newsroom/2019/09/iphone-11-pro-and-iphone-11-pro-max-the-most-powerful-and-advanced-smartphones/ — Apple Newsroom: iPhone 11 Pro announcement, September 10, 2019
https://blog.rust-lang.org/2019/07/04/Rust-1.36.0.html — Rust Blog: “Rust 1.36.0”, July 4, 2019
https://blog.rust-lang.org/2019/08/15/Rust-1.37.0.html — Rust Blog: “Rust 1.37.0”, August 15, 2019
https://keras.io — Keras documentation: François Chollet’s high-level API, integrated into TF 2.0
https://www.ft.com/content/b9bb4e54-dbc1-11e9-8f9b-77216ebe1f17 — Financial Times: Google quantum supremacy report, September 20, 2019
https://www.fast.ai — fast.ai: Jeremy Howard’s practical deep learning courses and ULMFiT methodology
https://karpathy.github.io — Andrej Karpathy’s blog: applied computer vision and autonomous systems
Free ebook
Free AI Video, Generated Locally
Run Wan 2.1 in ComfyUI on your own GPU — the scripts I use, measured times, sample clips. No cloud, no API keys.
No spam. Unsubscribe at any time.
Leave a Reply