2025-11-27 Tech Briefing | Native HTML tags can replace a lot of JS logic

Thursday

Welcome to this edition, featuring 43 articles.

📚 Contents/📝 TL;DR

📚 Contents

📝 TL;DR

Primary

These 5 Lesser-Known HTML Tags Can Save You 100 Lines of JS

Native HTML tags can replace a lot of JS logic

These 5 Lesser-Known HTML Tags Can Save You 100 Lines of JS

Native HTML tags can replace a lot of JS logic

Understanding Fiber through a Demo

An analysis of React Fiber's asynchronous rendering mechanism

Understanding Fiber through a Demo

An analysis of React Fiber's asynchronous rendering mechanism

Securing Amazon Elastic VMware Service (Amazon EVS) with AWS Network Firewall

AWS EVS and Firewall implement a hybrid cloud security architecture.

Securing Amazon Elastic VMware Service (Amazon EVS) with AWS Network Firewall

AWS EVS and Firewall implement a hybrid cloud security architecture.

NVIDIA Run:ai Falls Short; Huawei's Open-Source Solution Delivers

Huawei's Flex:ai open-source solution surpasses NVIDIA Run:ai in AI compute orchestration.

NVIDIA Run:ai Falls Short; Huawei's Open-Source Solution Delivers

Huawei's Flex:ai open-source solution surpasses NVIDIA Run:ai in AI compute orchestration.

AI for Ops vs. Ops for AI: Distinguishing Two Operational Paradigms

Distinguishing AI-enabled operations (AI for Ops) from AI system operations (Ops for AI)

AI for Ops vs. Ops for AI: Distinguishing Two Operational Paradigms

Distinguishing AI-enabled operations (AI for Ops) from AI system operations (Ops for AI)

How Letta Builds Production-Grade AI Agents Using Amazon Aurora PostgreSQL

Letta uses Aurora to enable AI agent persistence

How Letta Builds Production-Grade AI Agents Using Amazon Aurora PostgreSQL

Letta uses Aurora to enable AI agent persistence

New Token-Oriented Object Notation (TOON) Aims to Reduce Token Consumption and Lower the Cost of Large Language Models

New data format TOON can significantly reduce LLM token costs

New Token-Oriented Object Notation (TOON) Aims to Reduce Token Consumption and Lower the Cost of Large Language Models

New data format TOON can significantly reduce LLM token costs

Amazon EKS Introduces Provisioned Control Plane

AWS EKS launches Provisioned Control Plane to enhance performance of large-scale Kubernetes clusters.

Amazon EKS Introduces Provisioned Control Plane

AWS EKS launches Provisioned Control Plane to enhance performance of large-scale Kubernetes clusters.

Enhancing and Monitoring Network Performance for Machine Learning Inference on Amazon EKS

Introduces EKS container network observability, optimizing ML inference network performance.

Enhancing and Monitoring Network Performance for Machine Learning Inference on Amazon EKS

Introduces EKS container network observability, optimizing ML inference network performance.

Amazon EKS Blueprints for CDK Now Supports Amazon EKS Auto Mode

EKS Auto Mode integrates with CDK, simplifying Kubernetes management

Amazon EKS Blueprints for CDK Now Supports Amazon EKS Auto Mode

EKS Auto Mode integrates with CDK, simplifying Kubernetes management

Data Engineering Weekly #243

Kafka may be shifting entirely to a diskless architecture.

Data Engineering Weekly #243

Kafka may be shifting entirely to a diskless architecture.

Google Cloud Launches Tiered Storage for Bigtable

Google Cloud introduces tiered storage for Bigtable

Google Cloud Launches Tiered Storage for Bigtable

Google Cloud introduces tiered storage for Bigtable

Announcing Kyverno 1.16 Release

Kyverno 1.16 is released, with CEL policy beta version and enhanced Kubernetes policy management.

Announcing Kyverno 1.16 Release

Kyverno 1.16 is released, with CEL policy beta version and enhanced Kubernetes policy management.

Data Engineering Weekly #240

Data platforms and AI technologies are advancing rapidly.

Data Engineering Weekly #240

Data platforms and AI technologies are advancing rapidly.

Data Engineering Weekly #239

Updates on data engineering trends and practices.

Data Engineering Weekly #239

Updates on data engineering trends and practices.

Specification-Driven Development: The Waterfall Model Makes a Comeback

SDD is criticized as the 'waterfall model resurgence' in the AI era

Specification-Driven Development: The Waterfall Model Makes a Comeback

SDD is criticized as the 'waterfall model resurgence' in the AI era

Spring Recent News: GA Versions for Boot, Security, GraphQL, Integration, Modulith, Batch, etc.

Multiple Spring projects release GA versions, fully upgrading to the 7.0 technology stack.

Spring Recent News: GA Versions for Boot, Security, GraphQL, Integration, Modulith, Batch, etc.

Multiple Spring projects release GA versions, fully upgrading to the 7.0 technology stack.

Essential

Anthropic's 'AI Espionage' Revelation: Distinguishing Signals from Noise

Anthropic reveals Chinese-backed AI attacks, sparking security debates.

Anthropic's 'AI Espionage' Revelation: Distinguishing Signals from Noise

Anthropic reveals Chinese-backed AI attacks, sparking security debates.

The Era of Ten Thousand GPUs: What Kind of Computing Cluster Do Enterprises Need?

AI computing power demand is surging, and intelligent computing clusters have become a key solution

The Era of Ten Thousand GPUs: What Kind of Computing Cluster Do Enterprises Need?

AI computing power demand is surging, and intelligent computing clusters have become a key solution

Opening the Black Box of Financial Markets with AI: How Microsoft Asia Research Institute Builds Order-Level Simulation Engine MarS

Microsoft Asia Research Institute uses AI to build a financial order-level market simulation engine called MarS

Opening the Black Box of Financial Markets with AI: How Microsoft Asia Research Institute Builds Order-Level Simulation Engine MarS

Microsoft Asia Research Institute uses AI to build a financial order-level market simulation engine called MarS

How Apache Iceberg Addresses Data Lake Coordination Challenges

Apache Iceberg addresses data lake consistency, transaction, and evolution challenges.

How Apache Iceberg Addresses Data Lake Coordination Challenges

Apache Iceberg addresses data lake consistency, transaction, and evolution challenges.

Netflix Integrates Relational Database Infrastructure into Amazon Aurora, Achieving Up to 75% Performance Improvement

Netflix Achieves 75% Performance Improvement and 28% Cost Reduction

Netflix Integrates Relational Database Infrastructure into Amazon Aurora, Achieving Up to 75% Performance Improvement

Netflix Achieves 75% Performance Improvement and 28% Cost Reduction

Abandoning the 'Cascade' Architecture! Kuaishou OneRec Rebuilds the Recommendation System with Large Models, Reducing Service Costs to 1/10

Kuaishou OneRec rebuilds the recommendation system with large models, reducing costs by 1/10.

Abandoning the 'Cascade' Architecture! Kuaishou OneRec Rebuilds the Recommendation System with Large Models, Reducing Service Costs to 1/10

Kuaishou OneRec rebuilds the recommendation system with large models, reducing costs by 1/10.

Grafana Labs Releases Mimir 3.0, Introduces New Architecture for Enhanced Performance

Mimir 3.0 restructured architecture to achieve read-write separation, significantly improving performance and stability.

Grafana Labs Releases Mimir 3.0, Introduces New Architecture for Enhanced Performance

Mimir 3.0 restructured architecture to achieve read-write separation, significantly improving performance and stability.

The Pulse: Cloudflare Down / Gemini 3 Released / Antigravity AI IDE Test (With Login Solution)

Cloudflare outage, Gemini 3 release, and AI IDE testing.

The Pulse: Cloudflare Down / Gemini 3 Released / Antigravity AI IDE Test (With Login Solution)

Cloudflare outage, Gemini 3 release, and AI IDE testing.

Config-Driven Dynamic Agent Architecture Networks: Achieving Efficient Orchestration, Dynamic Updates, and Intelligent Governance

Deep dive into the agent architecture design paradigm

Config-Driven Dynamic Agent Architecture Networks: Achieving Efficient Orchestration, Dynamic Updates, and Intelligent Governance

Deep dive into the agent architecture design paradigm

Re-evaluating the Medallion Architecture: Data Vault in the Silver Layer, Dimensional Modeling in the Gold Layer

Best practices for Data Vault and dimensional modeling in the Medallion architecture.

Re-evaluating the Medallion Architecture: Data Vault in the Silver Layer, Dimensional Modeling in the Gold Layer

Best practices for Data Vault and dimensional modeling in the Medallion architecture.

Engineering Growth: The Data Layer in Modern GTM Systems

The GTM data layer is key to growth.

Engineering Growth: The Data Layer in Modern GTM Systems

The GTM data layer is key to growth.

Data Engineering Weekly #244

AI Agents are redefining enterprise system architecture.

Data Engineering Weekly #244

AI Agents are redefining enterprise system architecture.

Think Like a Data Engineer

The key to becoming an excellent engineer lies in your way of thinking.

Think Like a Data Engineer

The key to becoming an excellent engineer lies in your way of thinking.

Cloudflare Global Outage Traced to Internal Database Changes

Cloudflare experienced a global outage for nearly 3 hours due to database changes.

Cloudflare Global Outage Traced to Internal Database Changes

Cloudflare experienced a global outage for nearly 3 hours due to database changes.

When Dimensions Change Too Quickly, Iceberg is at a Loss

Traditional Iceberg struggles with rapidly changing dimensions; new architectures must balance mutability and immutability.

When Dimensions Change Too Quickly, Iceberg is at a Loss

Traditional Iceberg struggles with rapidly changing dimensions; new architectures must balance mutability and immutability.

General

Data Engineering Weekly #245

Data Engineering Weekly #245, focusing on AI agent patterns, Kafka controversy, LLM optimization, and big company practices.

Data Engineering Weekly #245

Data Engineering Weekly #245, focusing on AI agent patterns, Kafka controversy, LLM optimization, and big company practices.

Vercel Login Feature Now Fully Available

Vercel login feature GA, offering OAuth/OpenID authentication.

Vercel Login Feature Now Fully Available

Vercel login feature GA, offering OAuth/OpenID authentication.

AWS Private Certificate Authority Now Supports Partitioned CRLs

AWS Private CA supports partitioned CRL, enhancing certificate management capabilities for millions.

AWS Private Certificate Authority Now Supports Partitioned CRLs

AWS Private CA supports partitioned CRL, enhancing certificate management capabilities for millions.

Uno Platform 6.4 and Uno Platform Studio 2.0: .NET 10, VS2026, and Agentic AI

Uno platform updates support .NET 10 and AI-assisted development.

Uno Platform 6.4 and Uno Platform Studio 2.0: .NET 10, VS2026, and Agentic AI

Uno platform updates support .NET 10 and AI-assisted development.

Data Engineering Weekly #246

Data Engineering Weekly #246, covering AI data, Lakehouse, stream-batch integration, and performance optimization.

Data Engineering Weekly #246

Data Engineering Weekly #246, covering AI data, Lakehouse, stream-batch integration, and performance optimization.

Data Engineering Weekly #238

The Data Engineering Weekly includes updates on Dagster components, Iceberg, AI data preparation, real-time analytics, and more.

Data Engineering Weekly #238

The Data Engineering Weekly includes updates on Dagster components, Iceberg, AI data preparation, real-time analytics, and more.

How Supplyco Leverages Dagster for Real-Time Manufacturing Intelligence

Preview of a Dagster real-time data pipeline case study.

How Supplyco Leverages Dagster for Real-Time Manufacturing Intelligence

Preview of a Dagster real-time data pipeline case study.

Solving the Challenges of High Concurrency in the Gaming Industry: MongoDB's Fine-Grained Throttling and Lossless Migration Architecture | Tencent Cloud Database DBTalk

Promotional material, with no substantial technical content.

Solving the Challenges of High Concurrency in the Gaming Industry: MongoDB's Fine-Grained Throttling and Lossless Migration Architecture | Tencent Cloud Database DBTalk

Promotional material, with no substantial technical content.

Empowering the Ultimate Gaming Rollback Experience: Unveiling Tencent's Key-Based Flashback Technology | Tencent Cloud DBTalk

The article contains no substantial content and is merely a promotional piece for an event.

Empowering the Ultimate Gaming Rollback Experience: Unveiling Tencent's Key-Based Flashback Technology | Tencent Cloud DBTalk

The article contains no substantial content and is merely a promotional piece for an event.

Amazon EKS Blueprints for CDK Now Support Amazon EKS Auto Mode

EKS Auto Mode integrates with CDK, simplifying Kubernetes management.

Amazon EKS Blueprints for CDK Now Support Amazon EKS Auto Mode

EKS Auto Mode integrates with CDK, simplifying Kubernetes management.

Sponsor: Handbook of Data Platform Fundamentals Everyone Uses

Advertisement for an eBook on data platform basics.

Sponsor: Handbook of Data Platform Fundamentals Everyone Uses

Advertisement for an eBook on data platform basics.

Primary

These 5 Lesser-Known HTML Tags Can Save You 100 Lines of JS

Juejin Frontend Hottest•Published on November 25

Insight•Frontend Development•Score: 8/10

HTML tagsnative interactionaccessibilityperformance optimizationreduce JS

Source

◆

Understanding Fiber through a Demo

Juejin Frontend Hottest•Published on November 24

Insight•Software Architecture and Engineering Practices•Score: 8/10

React FiberAsynchronous RenderingTask SchedulingPerformance OptimizationEvent Loop

Source

◆

Securing Amazon Elastic VMware Service (Amazon EVS) with AWS Network Firewall

AWS - Architecture•Published on November 26

Insight•Cloud Computing and Backend•Score: 8/10

AWS EVSNetwork FirewallVMwareHybrid CloudNetwork Security

Source

◆

NVIDIA Run:ai Falls Short; Huawei's Open-Source Solution Delivers

InfoQ - Architecture•Published on November 26

Insight•AI and Frontier Technologies•Score: 8/10

Huawei Flex:aiRun:aiAI compute orchestrationGPU virtualizationopen source

🔬

Tech Insights

This analysis of Huawei's Flex:ai article is quite sharp, with several key points: 1. Directly challenging NVIDIA Run:ai: Huawei's bold comparison to Run:ai and claim of 'not done, but achieved' underscores its technical confidence and market ambition. Especially in open-source strategy and hardware compatibility, Huawei's comprehensive openness is a clear slap against NVIDIA's closed ecosystem. 2. Deep breakthrough in AI containers: Flex:ai is not just a scheduling tool, but a framework for building AI-native container infrastructure, managing at the NPU operator level to enable task-level preemption, migration, and fine-grained sharing. This is more thorough than Run:ai's upper-layer scheduling, offering better isolation and flexibility, particularly with XPU Pool and remote virtualization, which significantly enhance heterogeneous compute utilization. 3. Industry-focused deployment: Flex:ai clearly targets the pain points of small-scale AI clusters in industries like healthcare, solving issues of wasted compute resources, low concurrency, and scalability challenges. This shows Huawei's grasp of the key to AI proliferation—not everyone needs a super-large compute center, but everyone needs efficient and user-friendly 'small and elegant' solutions.

⚠️

Notable

Although Flex:ai's technical approach is very appealing, several points still require observation: 1. Ecosystem development: Open-source is the first step, but whether it can genuinely attract a large number of ecosystem partners and build a strong community like Kubernetes remains to be seen. After all, NVIDIA's developer habits and ecosystem stickiness built through CUDA are still formidable. 2. Performance stability: This deep-level virtualization inevitably comes with performance overhead and stability challenges. While the article mentions a '5%~10% improvement in inference performance', it sounds good, but whether it can maintain such performance in large-scale complex AI tasks needs more practical validation. After all, reliability in production environments matters more than theoretical data.

📈

Perspectives

At this point in time, Huawei's continued push in the AI compute infrastructure domain, especially with Flex:ai's open-source release, represents a strategic counterattack against Western vendors. As U.S.-China tech competition intensifies, domestic replacement becomes a trend. Huawei's approach of going deep into the stack while embracing open-source is sure to accelerate the process of self-reliant AI compute ecosystem development in China. For domestic enterprises, this means more diverse and flexible options in AI infrastructure selection, no longer entirely dependent on a single vendor. This not only reduces costs but also promotes AI application in more industry scenarios, especially for SMEs constrained by hardware and budget. Flex:ai offers a new cost-effective path. I believe that in the coming years, competition around AI compute orchestration will become even more intense, and Huawei has already revealed its key cards on this critical battlefield.

Source

◆

AI for Ops vs. Ops for AI: Distinguishing Two Operational Paradigms

Thoughtworks Tech•Published on November 27

Insight•AI Operations and Infrastructure•Score: 8/10

AIOpsOps for AIModel ObservabilitySRELLMOps

Source

◆

How Letta Builds Production-Grade AI Agents Using Amazon Aurora PostgreSQL

AWS - Databases•Published on November 26

Insight•AI and Frontier Technologies•Score: 8/10

AI agentpersistent memoryAurora PostgreSQLpgvector

Source

◆

New Token-Oriented Object Notation (TOON) Aims to Reduce Token Consumption and Lower the Cost of Large Language Models

InfoQ - Backend•Published on November 27

Insight•AI and Frontier Technologies•Score: 8/10

TOONLLMToken ConsumptionJSON AlternativeCost Optimization

Source

◆

Amazon EKS Introduces Provisioned Control Plane

AWS - Containers•Published on November 27

News•Cloud Computing and Backend•Score: 8/10

AWS EKSKubernetesProvisioned Control PlaneHigh PerformanceAI/ML Workloads

🔬

Tech Insights

1. Addressing Large-Scale K8s Pain Points: For clusters with tens of thousands of Pods, the stability and performance of the K8s control plane are core bottlenecks. Provisioned Control Plane eliminates performance fluctuations during sudden traffic spikes through pre-allocated capacity, targeting these extreme scenarios precisely. 2. Clear Performance SLA and Visibility: It offers multiple tiers such as XL, 2XL, and 4XL, and clearly defines performance metrics including API concurrency, Pod scheduling rate, and etcd database size, allowing users to have clear expectations of the control plane's capabilities. Monitoring is also provided via CloudWatch, making it very practical. 3. Reflection of Deep Technical Accumulation: AWS's ability to introduce such a service indicates deep optimization in the underlying architecture, storage, API, scheduler, and data plane of Kubernetes clusters. Particularly, support for 100,000 nodes and a new etcd architecture highlights its leadership in large-scale container management.

⚠️

Notable

1. Cost Barrier: This service is a 'paid option' and, although the article does not disclose specific pricing details, it is clearly targeted at customers with ample budgets and extreme performance requirements. Smaller businesses or cost-sensitive projects may perceive it as somewhat 'luxurious'. 2. Non-Universal Pain Points: The standard EKS control plane is already quite excellent, and its dynamic scaling capability is strong enough for most daily workloads. Provisioned Control Plane primarily addresses the special needs of 'top-tier' users (such as ultra-large AI/ML and extreme high concurrency), and is not suitable for all EKS users. 3. Inherent Limitations of 'Provisioned': While it resolves the lag in dynamic scaling, 'provisioned' itself implies the need for users to accurately predict peak demand, which can lead to capacity waste or under-provisioning if not done correctly, increasing operational complexity.

Source

◆

Enhancing and Monitoring Network Performance for Machine Learning Inference on Amazon EKS

AWS - Containers•Published on November 26

Insight•AI and Frontier Technologies•Score: 8/10

EKSML InferenceNetwork ObservabilityCloudWatchPerformance Optimization

Source

◆

Amazon EKS Blueprints for CDK Now Supports Amazon EKS Auto Mode

AWS - Containers•Published on November 26

Insight•Cloud Computing and Backend•Score: 8/10

EKS Auto ModeCDKKubernetesAutomated Management

Source

◆

Data Engineering Weekly #243

Data Eng Weekly•Published on October 27

Insight•Software Architecture and Engineering Practices•Score: 8/10

Kafka EvolutionFlink Real-time ProcessingData Replay TestingPrivacy InfrastructureLLM Post-training

Source

◆

Google Cloud Launches Tiered Storage for Bigtable

InfoQ - Cloud•Published on November 27

News•Cloud Computing and Backend•Score: 7/10

BigtableTiered StorageGoogle CloudCost OptimizationNoSQL

Source

◆

Announcing Kyverno 1.16 Release

CNCF•Published on November 26

Insight•Software Architecture and Engineering Practices•Score: 7/10

KyvernoKubernetesPolicy as CodeCELPolicy Management

Source

◆

Data Engineering Weekly #240

Data Eng Weekly•Published on October 6

News•Cloud Computing and Backend•Score: 7/10

Data platformLakehouseAIData qualityObservability

Source

◆

Data Engineering Weekly #239

Data Eng Weekly•Published on September 29

News•Cloud Computing and Backend•Score: 7/10

data trendsMLOpsAIdata flowdata governance

Source

◆

Specification-Driven Development: The Waterfall Model Makes a Comeback

InfoQ - AI & LLM•Published on November 27

Insight•Software Architecture and Engineering Practices•Score: 6/10

Specification-Driven DevelopmentSDDWaterfall ModelAgile DevelopmentAI Programming

Source

◆

Spring Recent News: GA Versions for Boot, Security, GraphQL, Integration, Modulith, Batch, etc.

InfoQ - Backend•Published on November 27

News•Software Architecture and Engineering Practices•Score: 6/10

Spring Boot 4.0Modular ArchitectureNull-Safe SupportAPI Version ControlSpring Modulith

Source

Essential

Anthropic's 'AI Espionage' Revelation: Distinguishing Signals from Noise

Thoughtworks•Published on November 26

Insight•AI and Frontier Technologies•Score: 9/10

AI securityAI jailbreakingenterprise attack surfaceAI warfareAI defense

Source

◆

The Era of Ten Thousand GPUs: What Kind of Computing Cluster Do Enterprises Need?

InfoQ - Cloud•Published on November 26

Insight•AI and Frontier Technologies•Score: 9/10

intelligent computing clusterAI computing powerelastic schedulingnetwork optimizationcomputing power supply

Source

◆

Opening the Black Box of Financial Markets with AI: How Microsoft Asia Research Institute Builds Order-Level Simulation Engine MarS

InfoQ - Big Data•Published on November 26

Insight•AI and Frontier Technologies•Score: 9/10

AI for FinanceFinancial Market SimulationOrder-Level DataAgentDigital Twin

🔬

Tech Insights

In my view, the most notable highlights of this article are threefold: 1. Order-Level Data Modeling: Compared to traditional models based on abstract datasets or expert features, MSRA directly tokenizes each order to build a close-to-reality and controllable order generation foundation model. This is akin to pulling the microstructure of financial markets into the open, representing a qualitative leap in precision and insight. 2. R&D Agent Automation Iteration: Through the collaboration of Research Agent and Development Agent, the quantitative research process is automated, enabling self-optimization and surpassing human expert designs. This 'AI alchemist' model is not only efficient but also significantly reduces reliance on expert manpower and knowledge, which is disruptive in the cost-sensitive financial sector. 3. Digital Twin Market MarS: Based on the order generation foundation model, a financial market Digital Twin simulation platform has been built. This means that extreme events like 'golden finger' can be safely simulated through data-driven methods for stress testing and what-if analysis. This is not merely backtesting, but an interactive and controllable future market sandbox.

⚠️

Notable

Although the results are promising, the article also highlights some challenges: 1. LLM Uncertainty: Financial scenarios require high levels of decision certainty, but LLM-based Agents inherently carry uncertainty. Bridging this contradiction is a major obstacle in actual deployment, requiring meticulous engineering and domain knowledge infusion. 2. Real-Time Bottleneck: The inference process for order flow Token sequences is more time-consuming than for natural language Token sequences. Predicting 5 minutes of data takes 15 minutes to generate, which is fatal in the fast-paced financial trading environment. Although it has been optimized to 1 minute, this delay is still unacceptable for high-frequency trading. This reminds us that even advanced models must pass the performance test. 3. Complexity of Integration: The system's integration with existing financial infrastructure and real-time data pipelines presents significant complexity, which could hinder scalability and real-world applicability.

📈

Perspectives

By November 2025, the enthusiasm for AI in finance will continue to rise. MSRA's work is undoubtedly a shot in the arm for the financial technology industry. It demonstrates that AI is not only applicable to risk control and intelligent customer service, but can also penetrate core trading, strategy optimization, and even market prediction. Particularly in China, securities firms and funds have a strong demand for cost reduction, efficiency enhancement, and strategy iteration, but high barriers, high risks, and regulatory compliance are unavoidable hurdles. Once this digital twin technology matures and ensures compliance, it will significantly reduce the cost of strategy experimentation and become a core competitive advantage for financial institutions. I believe this is more practical and has greater implementation potential compared to those applications that remain at the conceptual level.

Source

◆

How Apache Iceberg Addresses Data Lake Coordination Challenges

Thoughtworks•Published on November 26

Insight•Software Architecture and Engineering Practices•Score: 9/10

Apache IcebergData LakeData ConsistencyTransactionsLakehouse Integration

Source

◆

Netflix Integrates Relational Database Infrastructure into Amazon Aurora, Achieving Up to 75% Performance Improvement

AWS - Databases•Published on November 27

Insight•Cloud Computing and Backend•Score: 9/10

Amazon AuroraDatabase MigrationPerformance OptimizationCost Saving

Source

◆

Abandoning the 'Cascade' Architecture! Kuaishou OneRec Rebuilds the Recommendation System with Large Models, Reducing Service Costs to 1/10

InfoQ - AI & LLM•Published on November 27

Insight•AI and Frontier Technologies•Score: 9/10

Recommendation SystemLarge ModelKuaishou OneRecEnd-to-End ArchitectureComputational Power CostGenerative AI

🔬

Tech Insights

1. 'Cascade' to 'End-to-End': The core is to address the issues of feedback loops being difficult to close and low module consistency in traditional cascade architectures, which lead to extremely low machine utilization efficiency (MFU). OneRec covers the entire pipeline with a single model, significantly improving MFU, which is the fundamental transformation for cost reduction and efficiency gain. 2. Economic validation of large models in the recommendation field: The article provides rigorous cost analysis, proving that even 10B-level large models are profitable in commercial scenarios, thus challenging the common belief that 'the idea is good but not feasible' among many teams. 3. Semantic ID and Multimodal Representation: By converting sparse IDs into token sequences and combining with VLM to build an 'embedding that understands both content and recommendations', the limitations of traditional recommendation systems in content understanding are overcome, enhancing deep semantic matching in recommendations.

⚠️

Notable

1. High Implementation Threshold: As a leading company, Kuaishou has substantial technical resources and massive data to support the training and deployment of such large models. For small and medium enterprises, the difficulty and cost of reproducing an end-to-end large model recommendation system are almost unbearable. 2. Challenge of 'Time Leakage': The article mentions that the 'user Doc + next action prediction' paradigm is not feasible in recommendation scenarios due to the difficulty in addressing the time leakage problem. Although Kuaishou resolved this by decomposing samples, it shows that the paradigm shift to large models in recommendations is not smooth and many details still need to be tackled. 3. Maturity of OneRec-Think: Although OneRec-Think (a multimodal generative understanding unified foundation) shows promising potential, it is currently 'not yet fully surpassing the online system' and 'costs are still high, making it unsuitable for full-scale online deployment', indicating that the technology's maturity still needs to be observed.

Source

◆

Grafana Labs Releases Mimir 3.0, Introduces New Architecture for Enhanced Performance

InfoQ - Architecture•Published on November 27

Insight•Software Architecture and Engineering Practices•Score: 9/10

Mimir 3.0Read-Write Separation ArchitectureStreaming Query EngineObservabilityPromQL Compatibility

Source

◆

The Pulse: Cloudflare Down / Gemini 3 Released / Antigravity AI IDE Test (With Login Solution)

Data Eng Weekly•Published on November 19

Insight•AI and Frontier Technologies•Score: 9/10

CloudflareGemini 3AntigravityLarge ModelsAI IDE

Source

◆

Config-Driven Dynamic Agent Architecture Networks: Achieving Efficient Orchestration, Dynamic Updates, and Intelligent Governance

Data Eng Weekly•Published on September 25

Insight•AI and Frontier Technologies•Score: 9/10

Agent ArchitectureConfiguration-DrivenDynamic UpdatesA2A ProtocolRAG Knowledge Base

Source

◆

Re-evaluating the Medallion Architecture: Data Vault in the Silver Layer, Dimensional Modeling in the Gold Layer

Data Eng Weekly•Published on October 17

Insight•Software Architecture and Engineering Practices•Score: 9/10

Medallion ArchitectureData VaultDimensional ModelingData WarehouseData Lake

Source

◆

Engineering Growth: The Data Layer in Modern GTM Systems

Data Eng Weekly•Published on October 7

Insight•Software Architecture and Engineering Practices•Score: 9/10

GTMdata governancedata engineeringprivacy protectiondata collaboration

Source

◆

Data Engineering Weekly #244

Data Eng Weekly•Published on November 3

Insight•AI and Frontier Technologies•Score: 9/10

AI Agent ArchitectureLLM Reasoning CostData Platform GovernanceUser Memory SystemSchema Version Control

Source

◆

Think Like a Data Engineer

Data Eng Weekly•Published on October 23

Insight•General Information•Score: 9/10

Systematic ThinkingData ModelingCareer GrowthMindsetIterative Design

Source

◆

Cloudflare Global Outage Traced to Internal Database Changes

InfoQ - Cloud•Published on November 27

News•Comprehensive News•Score: 8/10

Cloudflare outagesingle point of failuremulti-cloud strategyconfiguration changechaos engineering

Source

◆

When Dimensions Change Too Quickly, Iceberg is at a Loss

Data Eng Weekly•Published on September 16

Insight•Software Architecture and Engineering Practices•Score: 8/10

IcebergRapidly Changing DimensionsData LakehouseHudiPaimon

🔬

Tech Insights

The value of this article lies in its direct address of a core pain point in data lakehouse architecture, offering clear analysis and multiple solution directions: 1. Rapidly changing dimensions (FCDs) challenge: The article clearly defines the characteristics of FCDs (high update frequency, low latency, time correctness, etc.) and highlights their fundamental conflict with Iceberg's 'immutable append' design philosophy. This directly challenges those who claim 'data lakes are everything,' reminding us there is no one-size-fits-all solution. 2. Limitations of existing Iceberg solutions: It thoroughly analyzes issues such as poor read performance with Equality Deletes, difficulty in implementing Positional Deletes, high latency with Buffered MERGE, and the inability of Medallion Layering to resolve core problems. This provides architects with a clear understanding when making selection decisions, avoiding blind following trends. 3. Rise of LSM tree solutions: Hudi and Paimon, as write-optimized solutions, leverage LSM trees and Delta Logs to achieve true Upsert and incremental querying, making them powerful tools for addressing FCDs. This reaffirms that in certain scenarios, write-optimized data formats are indispensable. 4. Trend toward hybrid architectures: The article proposes multiple approaches such as Federated OLAP-Lakehouse, external index-enhanced Iceberg, and DuckLake, indicating that the industry is actively exploring hybrid architectures to meet complex and evolving data needs, rather than sticking to a single technology stack.

⚠️

Notable

While the article is comprehensive, it could provide more critical insight into the challenges of Hudi and Paimon in terms of ecosystem maturity, community activity, and integration with mainstream toolchains. After all, technology selection isn't just about the technology itself; it's also about 'whether there are enough people and whether the pitfalls are deep enough'. Federated OLAP-Lakehouse, although capable of solving problems, carries significant operational complexity and cost for maintaining data consistency. For most domestic companies lacking top-tier data teams, this is akin to 'suicidal' innovation. DuckLake's approach of metadata transactionalization is intriguing, but its maturity, large-scale production use cases, and community support remain unclear as of November 2025, making it risky to adopt prematurely.

📈

Perspectives

By November 2025, domestic companies have reached a critical point in their demand for real-time capabilities in data lakes. Those still using Iceberg for Batch ETL and claiming to be 'real-time data lakes' are essentially deceiving themselves. The market is calling for genuine 'real-time lakehouse' solutions. Hudi and Paimon are expected to gain increasing attention in China, particularly in industries such as e-commerce and financial risk control that require extremely high real-time performance. I believe the future data platforms will be 'multi-format, multi-engine, and hybrid architectures'. No single format can meet all needs, and architects must flexibly combine Iceberg (for historical data), Hudi/Paimon (for real-time updates), and OLAP databases (for fast querying) based on business scenarios. Those that only promote a single product will find their market shrinking, and vendors should consider how to provide lower operational costs and simpler hybrid architecture management—this is what the domestic market most urgently needs for 'cost reduction and efficiency enhancement'.

Source

General

Data Engineering Weekly #245

Data Eng Weekly•Published on November 10

News•Comprehensive News•Score: 7/10

AI AgentData PlatformStream ProcessingKafkaLLM Inference

🔬

Tech Insights

AI Agent Patterns: The article summarizes seven core design patterns, including Prompt Chaining, Routing, and Tool Use, providing a clear framework and guidance for building scalable and adaptive AI agent systems. This is a valuable 'secret recipe' for teams exploring AI Agent implementation. Kafka vs. Postgres Controversy: It clearly states that Postgres cannot replace Kafka in event stream processing, low-latency streaming, and connector ecosystem, emphasizing the principle of each tool having its own role. This helps teams avoid the 'silver bullet' fallacy. Event Stream Market Saturation: Rumors about Confluent's sales have sparked discussions on the market saturation of event streams, pointing out the impact of cloud-native solutions (like MSK) on open-source business models. This indicates a business model transformation in the infrastructure domain. LinkedIn LLM Inference Acceleration: By using speculative decoding (n-gram prediction), LinkedIn achieved nearly 4x higher inference throughput and a 66% reduction in P90 latency, showcasing extreme practice in LLM inference optimization. This has high reference value for teams aiming to enhance AI application performance. Pinterest and Netflix AI/ML Platform Evolution: The article reveals the complexity and value of building mature AI platforms as big companies have spent a decade evolving from fragmented systems to unified platforms, incorporating GPU and Ray systems, and optimizing development experiences with Metaflow.

⚠️

Notable

'You don't need Kafka, just use Postgres' is a reflection of the industry's lack of deep thinking in technical selection. There's always someone who wants to use a single tool for everything, leading to either overcomplicating simple problems or failing at complex ones. This article serves as a cold shower for those who believe in the 'silver bullet' theory. The discussion on the 'saturation' of the event stream market highlights the challenges faced by open-source infrastructure companies in China. If they cannot offer unique value or services that cloud vendors cannot match, they will eventually be dragged into price wars or replaced by cloud vendors' 'free' offerings. Business model innovation is more urgent than technological innovation. The experiences of big companies in AI/ML platforms provide strong reference for domestic enterprises building their own AI infrastructure. Systematic and platform-based construction is the only way to enhance AI productivity. However, many domestic companies are still re-inventing the wheel, lacking long-term planning and platform thinking.

📈

Perspectives

In my view, AI agents and LLM inference optimization are currently the hottest areas. Domestic big companies and startups are all competing for this pie. LinkedIn's practices demonstrate that performance optimization in LLM applications still has significant potential. The 'market saturation' argument for event streams serves as a warning for domestic open-source infrastructure companies. If they cannot provide unique value or services that cloud vendors cannot match, they will inevitably be dragged into price wars or replaced by cloud vendors' 'free' offerings. Business model innovation is more urgent than technological innovation. The experiences of big companies in AI/ML platforms provide strong reference for domestic enterprises building their own AI infrastructure. Systematic and platform-based construction is the only way to enhance AI productivity. However, many domestic companies are still re-inventing the wheel, lacking long-term planning and platform thinking.

Source

◆

Vercel Login Feature Now Fully Available

Vercel News•Published on November 26

News•Frontend Development•Score: 6/10

VercelOAuthOpenIDAuthenticationDeveloper Tools

Source

◆

AWS Private Certificate Authority Now Supports Partitioned CRLs

AWS - Security•Published on November 26

Insight•Cloud Computing and Backend•Score: 6/10

AWS Private CACRLPKICertificate RevocationScalability

Source

◆

Uno Platform 6.4 and Uno Platform Studio 2.0: .NET 10, VS2026, and Agentic AI

InfoQ - Backend•Published on November 27

Insight•Software Architecture and Engineering Practices•Score: 6/10

Uno Platform.NET 10Agentic AICross-Platform UIVisual Studio 2026

Source

◆

Data Engineering Weekly #246

Data Eng Weekly•Published on November 17

News•Comprehensive News•Score: 6/10

Data EngineeringAI AgentData LakeStream ProcessingPerformance Optimization

🔬

Tech Insights

Dagster Compass: AI data analysts integrated into Slack, which is a novel approach. It provides AI-driven data questioning directly within workflows, eliminating the need for dashboards and ticketing processes, enhancing decision-making efficiency. Lakshmanan discusses AI data preparation, emphasizing the shift from heavy standardization to context-rich, curated examples, and building AI agent infrastructure, pointing out the core paradigm shift in AI-era data engineering. Iceberg stream-batch integration: A comparison between Apache Fluss and Confluent Kora/TableFlow, illustrating the ongoing industry exploration of merging Lakehouse with stream processing, addressing data timeliness pain points. Single-machine performance optimization: Benchmark testing of DuckDB, Polars, and other tools on 650GB data proves that in certain scenarios, single-machine processing can outperform distributed frameworks, offering reference value for teams with limited resources or seeking extreme efficiency. Netflix's knowledge graph and real-time distributed graph: Demonstrates how Netflix uses knowledge graphs and KVDAL (based on Cassandra) to unify and access massive heterogeneous entertainment data, serving as a paradigm for massive-scale data management.

⚠️

Notable

As a weekly report, it's more of a collection of various topics. The content is abundant but superficial, touching on each subject only briefly. To gain deeper insights, one must dig into the original sources. It's like an all-you-can-eat buffet, rich in variety but lacking a main course. Dagster Compass, an AI data analysis tool, sounds impressive, but how effective is it in practice? Accuracy, hallucination issues, and the ability to understand complex business logic are all areas that need scrutiny. While single-machine performance testing is impressive, don't forget that distributed frameworks offer scalability and fault tolerance, which not all scenarios can replace. Overemphasizing single-machine performance might lead some junior teams to misjudge the boundaries of technical selection. IBM's CDO report states that 26% of CDOs lack confidence in the value of unstructured data? This data, viewed in November 2025, is simply 'the Emperor's New Clothes'. Large models have been popular for two years, and unstructured data is the lifeblood of AI. This confidence index seems too low, either due to flawed survey samples or overly conservative CDOs.

📈

Perspectives

In my view, this report reflects several core trends in the data engineering field: the reshaping of data infrastructure by AI, the ongoing evolution of data lake architectures, and the pursuit of extreme performance and efficiency. The rise of AI data preparation and AI agents indicates that the role of data engineers is shifting from traditional ETL pipeline builders to 'feeders and trainers' of AI models. Those who can better serve AI will gain a competitive edge in the market. The emergence of high-performance single-machine tools also presents 'a shortcut for catching up' for domestic startups. Not all businesses require big data solutions; many small and medium enterprises need cost-effective and easy-to-deploy solutions. Tools like DuckDB and Polars have significant application potential in China. Large companies' technical sharing (Netflix, Uber) remains a benchmark for the industry, offering 'massive-scale, real-time, and high availability' insights that domestic leading companies can learn from.

Source

◆

Data Engineering Weekly #238

Data Eng Weekly•Published on September 22

News•Comprehensive News•Score: 6/10

Data EngineeringIcebergAI-Ready DataReal-Time AnalyticsLLM Evaluation

🔬

Tech Insights

As an architect, I focused on several key points in this week's report: 1. Dagster Components GA: This general availability means we can build data pipelines with less boilerplate code. It's a good step toward improving development efficiency and lowering the data engineering barrier, but real-world large-scale adoption will depend on the maturity of the community. 2. The Six Pillars of AI-Ready Data: The article emphasizes that AI data is not only high-quality but also requires context understanding, cross-warehouse unification, real-time accessibility, etc. This hits a key pain point for many domestic companies during AI implementation — data is the foundation of AI, but is often underestimated; everyone is chasing models, but forgetting the base. 3. LLM Evaluation Guide: At this point in November 2025, LLM applications have entered deeper waters, and the maturity of the evaluation system is a key determinant of whether a product can be deployed. This guide provides a systematic evaluation framework, including human review, automated metrics, and the LLM-as-judge approach, which is very practical. 4. Skepticism Toward the MCP Protocol: Chris Riccomini's article reiterates that the Model Context Protocol (MCP) is just a re-invention of existing OpenAPI/gRPC. This aligns with our previous views, indicating that the industry still has controversies over the future of AI Agent protocols, far from forming a unified standard.

📈

Perspectives

In November 2025, the domestic data engineering field is still racing along the two main tracks of 'real-time' and 'AI'. Whether it's the real-time challenges of Iceberg, Stripe's real-time analytics, or 'AI-ready data', they all point toward these two directions. Domestic data platform and tool providers that still emphasize traditional batch processing capabilities are basically out of the game. Now, the competition is about end-to-end real-time capabilities and deep adaptation to AI scenarios, including LLM evaluation and agent data interaction. The ongoing skepticism toward MCP indicates that the 'middleware' or 'protocol layer' for AI agents has yet to form a unified standard, which is both a challenge and an opportunity. Whoever defines the next-generation agent interaction protocol will gain the upper hand. So far, the phenomenon of 'reinventing the wheel' remains prevalent, and the industry needs deeper innovation rather than simple packaging.

Source

◆

How Supplyco Leverages Dagster for Real-Time Manufacturing Intelligence

Data Eng Weekly•Published on October 20

News•Software Architecture and Engineering Practices•Score: 4/10

Dagsterreal-time datamanufacturing intelligence

Source

◆

Solving the Challenges of High Concurrency in the Gaming Industry: MongoDB's Fine-Grained Throttling and Lossless Migration Architecture | Tencent Cloud Database DBTalk

InfoQ - Big Data•Published on November 26

News•Software Architecture and Engineering Practice•Score: 1/10

high concurrencythrottlinglossless migrationMongoDBgame architecture

Source

◆

Empowering the Ultimate Gaming Rollback Experience: Unveiling Tencent's Key-Based Flashback Technology | Tencent Cloud DBTalk

InfoQ - Big Data•Published on November 26

News•General Information•Score: 1/10

Tencent CloudMongoDBFlashback TechnologyDBTalk

Source

◆

Amazon EKS Blueprints for CDK Now Support Amazon EKS Auto Mode

InfoQ - Big Data•Published on November 26

News•Cloud Computing and Backend•Score: 1/10

EKS Auto ModeCDKKubernetesAutomation Management

Source

◆

Sponsor: Handbook of Data Platform Fundamentals Everyone Uses

Data Eng Weekly•Published on October 13

News•General Information•Score: 1/10

Data PlatformE-bookAdvertisement