Processing ads information and billions of events at a microsecond scale requires distributed systems that are scalable, reliable and privacy-safe in today's world.
The processing of ads information and billions of events at a microsecond scale requires distributed systems that are scalable, reliable and privacy-safe in today's world. I'll share some of the challenges of building these large-scale distributed systems, which are also applicable to general ads processing.
These systems operate at a scale of milliseconds and sometimes microseconds. Shaving off even those few precious moments can help provide data to downstream consumers much faster—thus improving the quality of future ads that are delivered. The largest costs in terms of latency come from external calls to storage systems . It's important to re-analyze those and then parallelize and optimize them.Events that are ready for processing are in the millions—and sometimes billions—per minute or hour. The design of these systems must allow them to be ready to scale the processing to a huge number of machines, and characteristics such as sharding and parallel processing are super critical. To scale the systems now or somewhere down the line, it's important to get an extensible sharding mechanism.The systems must care about reliability as a fundamental aspect. Systems go down for maintenance and other unknown real-time reasons, which means we must create redundancy and protection under failure scenarios. Build layers of dependency with datacenter support for multiple regions and high availability systems that can take over the entire shift of traffic when system or network failures happen.To understand the scale and failures, we have to analyze the data in real time. Though a topic in itself of how to build large-scale analytics platforms, we must build real-time analytics to slice and dice the data we receive. Building an analytics framework and integrating it fully into all of the services and pieces of the stack can help bring out various aspects and inefficiencies that might be preventing the stack from being fully optimized.on how ads data can and can't be used based on the various changes. We must have the ability to flow relevant information as metadata to distinguish the various slices. This allows us to make correct decisions in processing, such as filtering and routing. Having frameworks that provide silos inside services to ensure data such as personally identifiable information isn't leaked can help maintain privacy standards.Systems need to be able to route the data to other systems. To do this, we must build frameworks that can route billions of events to the right subsequent systems. One incorrect decision could send events the wrong way, which would be impossible to fix. Manual routing at a large scale is not recommended due to the various edge cases, and a nonscalable approach can lead to huge failure outcomes. Having a routing framework that is integrated across the stack can provide a holistic approach to this problem.Due to the scale of these events, the efficiency of systems becomes critical. There is always a lack of power to keep the systems running. Optimizations are vital across the entire stack. Small optimizations can also lead to huge power and capacity reductions as well as efficiency wins. Given the power crunch, this will continue to remain a bottleneck in building continuously larger systems.We just don't build for one layer of systems to handle one kind of processing. We need to be able to build for layers of systems, each of which handles the independent processing of events. We must add buffering for each layer to prevent any layer from being overwhelmed during the process. Buffering frameworks such as Kafka and Apache Flink can hold massive amounts of data between different processing steps to prevent overwhelming systems.Systems across layers need to communicate with each other. Establishing protocols that allow systems to transfer information is crucial. Any bugs can disrupt the entire processing of the stack. Ad hoc communication mechanisms only lead to bugs. Use tested remote procedure call mechanisms that are highly efficient, deployable and provide various cross-language support.Building a large-scale testing toolkit to prevent bugs during production is essential. This includes testing individual systems as well as the mesh of systems to ensure one system doesn't corrupt another downstream system in any possible way. Various levels of testing, such as unit, pre-production and integration testing, are critical to prevent bad deployments. Though these are a subset of challenges, we should expect most of them to fall in the above bucket of issues. Building and maintaining massive-scale systems is extremely challenging, and it's vital to have the right principles as a foundation to scale the complex systems. Now is the time to build these right.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
This TV Soundbar Now Costs 6x Less Than Bose, Amazon Just Sold 7K UnitsThe Amazon Fire TV Soundbar delivers virtual surround sound with Dolby Audio processing.
Read more »
Qualcomm Readies Rack-Scale AI With New Chips And RoadmapQualcomm announces new chips, cards and rack-scale solutions for the AI data center, but omits key details.
Read more »
CEO explains multi-million investment in US rare earth processing plant, makes bold 2028 predictionAclara Resources CEO Ramón Barúa Costa addresses how his company is joining the U.S.' push for rare earth independence from China on 'The Claman Countdown.'
Read more »
CEO explains multimillion-dollar investment in US rare earth processing plant, makes bold 2028 predictionAclara Resources CEO Ramón Barúa Costa addresses how his company is joining the U.S.' push for rare earth independence from China on 'The Claman Countdown.'
Read more »
Qualcomm Challenges Nvidia And AMD With Data Center AI ChipsQualcomm’s transition from consumer devices to enterprise-grade AI infrastructure exemplifies not just a rebalancing in hardware competition, but a redefinition in how business value is realized from AI investments.
Read more »
Qualcomm Challenges Nvidia And AMD With Data Center AI ChipsQualcomm’s transition from consumer devices to enterprise-grade AI infrastructure exemplifies not just a rebalancing in hardware competition, but a redefinition in how business value is realized from AI investments.
Read more »