Hpc (high Performance Computing)

High Performance Computing most generally refers to the practice of aggregating computing power in a way that delivers much higher performance than one could get out of a typical desktop computIn simple terms, HPC enables us to first model then manipulate those things that are important to us. HPC changes everything. It is too important to ignore or push aside. Indeed, HPC has moved from a selective and expensive endeavor to a cost-effective technology within reach of virtually every budget. HPC is actually used in two ways: it can either mean “high performance computing” or “high performance computer.” It’s usually pretty clear from the context which sense is being used. To many organizations, HPC is now considered an essential part of business success. Your competition may be using HPC right now. They won’t talk much about it because it’s considered a competitive advantage. Of one thing you can be sure, however; they’re designing new products, optimizing manufacturing and delivery processes, solving production problems, mining data, and simulating everything from business process to shipping crates all in an effort to become more competitive, profitable, and “green.”

HPC may very well be the new secret weapon. You may have heard of supercomputing, and monster machines from companies like Cray and IBM, that work on some of mankind’s biggest problems in science and engineering. Origins of the universe, new cancer drugs, that sort of thing. These are very exotic machines by virtue of the technologies inside them, and the scale at which they are built: sometimes 10,000 of thousands of processors make up a single machine. For this reason supercomputers are expensive, with the top 100 or so machines in the world costing upwards of $20M each. This kind of computing is related to the HPC you might consider for your business in the way that Formula One racers are related to your Camry. They are both cars, but that’s about where the similarity ends. Supercomputers, like race cars, take vast sums of money and specialized expertise to use, and they are only good for specialized problems (you wouldn’t drive a race car to the grocery store).

But a high performance computer, like the family sedan, can be used and managed without a lot of expense or expertise. If you’ve never done this before, you will need to learn new things. An HPC machine is more complex than a simple desktop computer — but don’t be intimidated! The basics aren’t that much more difficult to grasp, and there are lots of companies (big and small) out there that can provide as much or as little help as you need. High performance computers of interest to small and medium-sized businesses today are really clusters of computers. Each individual computer in a commonly configured small cluster has between one and four processors, and today’s processors typically have from two to four cores. HPC people often refer to the individual computers in a cluster as nodes. A cluster of interest to a small business could have as few as four nodes, or 16 cores. A common cluster size in many businesses is between 16 and 64 nodes, or from 64 to 256 cores. The point of having a high performance computer is so that the individual nodes can work together to solve a problem larger than any one computer can easily solve. And, just like people, the nodes need to be able to talk to one another in order to work meaningfully together. Of course computers talk to each other over networks, and there are a variety of computer network (or interconnect) options available for business cluster (see here for an overview of cluster interconnects).

Hpc Within Weather Forecasting

Weather forecasts focus on short-term conditions while climate predictions focus onlong-term trends. You dress for the weather in the morning; you plan your winter vacation in the Virgin Islands for the warm, sunny climate. Delivering weather forecasts multiple times per day demands a robust computing infrastructure and a 24x7x365 focus on operational resilience. Computing a weather forecast requires scheduling a complex ensemble of pre-processing jobs, solver jobs and post-processing jobs. Since there is no use in a forecast for yesterday, the prediction must be delivered on time, every time. The best practice to deliver forecasts is to deploy two identical supercomputers, each capable of producing weather forecasts by itself. This ensures a backup is available if one system goes down. Running climate predictions are longer term jobs requiring bigger computations.There is no immediate risk if a climate computation takes a bit longer to run. Generally, climate prediction shares access to spare cycles on high-performance computing (HPC) systems whose first priority is weather. Fitting in and sharing access without impacting weather forecasting are the priorities. This complex prioritization and scheduling of weather simulations and climate predictions on HPC system sis where Altair’s PBS Professional (PBS Pro) workload management technology plays a keyrole. It is a natural fit for HPC applications such as weather and climate forecasting.

It enables users to address operational challenges such as resource conflicts due to more concurrent high-priority jobs, complexity of mixed operational and research workload, and unpredictability of emergency or other high-priority jobs. PBS Pro supports advance and recurring reservations for regular activities such as forecasts. It also provides automatic fail over and a 100% health check framework to detect and mitigate faults before they cause problems. Flexible scheduling policies mean top priority jobs (forecasts) finish on time while secondary-priority jobs (climate predictions) are fit in to maximize HPC resource utilization. In addition, the PBS Plugin Framework offers an open architecture to support unique requirements. Users can plug in third-party tools and even change the behavior of PBS Pro.

History of Weather Forecasting Through Hpc

The practice of meteorology can be traced as far back as 3000 BC in India. Predicting the weather via computation, though comparatively recent, pre-dates electronic computers. In 1922, British mathematician Lewis Fry Richardson posited employing a whopping 64,000 human computers (people performing computations by hand) to predict the weather – 64,000 “computers” were needed to per form enough calculations, quickly enough, to predict the weather in “real time” (humans calculate at about 0.01 FLOPS, or FLoating-point Operations Per Second). It took another 30 years for electronic computers to produce the first real forecast: In 1950, ENIAC computed at a speed of about 400 FLOPS and was able to produce a forecast 24 hours in advance – in just under 24 hours – making it a match for Richardson’s 64,000 human computers in both compute speed and timeliness of result. Forecasts are created using a model of the earth’s systems by computing changes based on fluid flow, physics and chemistry. The precision and accuracy of a forecast depend on the fidelity of the model and the algorithms, and especially on how many data points are represented. In 1950, the model represented only a single layer of atmosphere above North America with a total of 304 data points. Since 1950, forecasting has improved to deliver about one additional day of useful forecast per decade (a four-day forecast today is more accurate than a one-day forecast in 1980). This increase in precision and accuracy has required huge increases in data model sizes (today’s forecasts use upwards of 100 million data points) and a commensurate, almost insatiable, need for more processing power. Today, weather centers across the globe are investing in petascale HPC to produce higher resolution and more accurate globaland regional weather predictions.

Future Developments

Supercomputing, along with big data, can meet the future demands of weather forecasting in three key areas:

1. Managing and utilizing enormous data sets: The volume and diversity of environmental data is increasing exponentially, placing great demand on the infrastructure to transport, manage, and store this data, and requiring ever-greater computational power for simulations that use it. This creates new opportunities for specialized services, developed with researchers in public and private institutions. One example is leveraging new sources of observation, such as sensors placed on automobiles. Imagine thousands of sensors in an urban area providing real-time meteorological information. Models are also evolving to analyze this tsunami of data and augment traditional physics-based simulations.
2. Increasing model resolution: Higher-resolution models are a critical element to better estimate the long-term state of climate systems and to improve weather forecasting, particularly for severe weather events. Recent simulations of Hurricane Sandy by researchers at the National Center for Atmospheric Research and the University of Illinois using Blue Waters supercomputers have zeroed in to a 500-meter resolution — the equivalent of a few city blocks.
3. Addressing technology hurdles: As weather modeling and analytics become more data-intensive and computationally demanding, researchers must watch for performance bottlenecks such as memory, I/O, and interconnect latencies and bandwidths. Weather simulation requires thousands of microprocessors to run in parallel, pushing hardware and software to its scalability limits. In addition, scalable operating systems, compilers, and application libraries play an essential role in achieving sustained performance. Ultimately, the underlying technology infrastructure must be tightly integrated to support simulation and analytics workflows.

Infrastructures offering simulation and data-driven analytics capabilities to support routine execution of high-resolution forecasts will combine with advanced research to promote a whole new array of specialized meteorological services for public and private sectors. The future of weather forecasting requires capabilities we couldn’t even conceive of when we began predicting the weather 64 years ago. Supercomputing innovation has so far kept pace with the demands of the community, and it is poised to offer new solutions in the years to come.