e2e - Information provided by Azul's engineers

REALTime Performance Monitor (RTPM)

RTPM is a tool for analyzing the performance and behavior of Java applications on-the-fly and in production environment. To learn more, watch a recorded performance debugging session, or read the data sheet provided below.

RTPM Demonstrations

• CPU bottlenecks explored
• Locking bottlenecks explored

• IO bottlenecks explored

REALTime Performance Monitor

REALTime Performance Monitor (RTPM)
RTPM is a performance analysis and debugging tool built into the Azul JVM. RTPM allows users to investigate Java application behavior on-the-fly using a standard web browser.

• Threads View. Browse what threads are doing, and whether they are working or waiting (and if so, for how long). See an instantaneous stack trace snapshot for all threads.

• CPU Usage Profiler. Identify execution hotspots as candidates for optimization, see activity level of application or JVM components over time.

• Hot-locks. Spot expensive locks causing your threads to block and preventing multicore scaling.

• System call profile. Identify system calls that occur frequently or take a long time to complete.

• No-setup. RTPM requires no configuration and no code changes.

• Lightweight and always-on. RTPM information is collected nonstop with minimal execution impact, even when you're not looking at it. It can be confidently used on production applications without stability or performance concerns. RTPM gives IT organizations the power to immediately analyze deployed applications under real-world loads when a problem occurs.

Download the RTPM datasheet »

Download the Azul Compute Appliance datasheet »

Introduction to Performance Tools

You won't learn much from this one, but it's cool :-)

Conference Presentations

Gil Tene
CTO
Azul Systems
gil@azulsystems.com

Dr. Cliff Click
Chief JVM Architect, Distinguished Engineer
Azul Systems
cliffc@azulsystems.com

Michael Wolf
Distinguished Engineer
Azul Systems
wolf@azulsystems.com

Pausless Garbage Collection

Current Java platforms invariably bring applications to a complete halt to clean up memory space, driving transaction response times above expected levels and compromising quality of service. Azul Compute Appliances eliminate virtually all application pauses associated with garbage collection by using a unique hardware-assisted algorithm. Pauseless Garbage Collection is concurrent to the application, always able to distribute free memory to threads; and is parallel, scaling to additional available cores to keep up with soaring allocation rates. The elimination of unruly pauses means applications can deliver consistent, predictably fast response times and drive more revenue through improved SLAs.

Performance Considerations in Concurrent Garbage-Collected Systems »
The presentation discusses and explains relevant GC terminology and phrases common in concurrent and mostly concurrent GC, focusing on their effects and relationship to metrics such as heap size, real and effective live set size, and object allocation rates. These include concurrent and mostly concurrent marking, live set and card marking, generational operation, and compaction.

On Fragmentation & Compaction

As java programs operate, they continually allocate objects of varying sizes in the heap in order to perform useful work. As some objects age and die, empty “dead” spaces appear in the heap at a rate that matches the allocation rate, and these spaces must be reclaimed and reused for use by newly allocated objects in order to sustain program execution.

Garbage collection, in all forms, deals with locating, reclaiming, and reusing these empty spaces. This reuse has two main possible forms:

1. In-place reuse: Empty spaces in the heap are identified and tracked (usually in “free lists”), and a new object is allocated which can fit into an empty space, the space is used to accommodate the new object.

2. Compaction: Empty spaces are reclaimed by moving live objects around such that large contiguous empty ranges in memory become available. Allocation is performed linearly into these large contiguous blocks, with objects of varying sizes packed together back-to-back during allocation.

While the use of in-place-reuse can be effective for delaying compaction, the heap gets continually fragmented as objects of varying sizes come and go. Eventually, there will come a time where many small empty spaces are available, but a single object that is larger than each of the individual spaces cannot be allocated without de-fragmenting and compacting the heap.

Fragmentation, and the resulting need for compaction, is inevitable. This is best evidenced by the fact that every commercial garbage collector implementation in the enterprise java world currently includes significant amounts of code that performs Heap Compaction.

Compaction and Pauses

Compaction can be a problematic operation for many Java Virtual Machines. Compaction requires live objects to be moved form one location to another in memory, and as a result all references to those objects must be tracked down and safely remapped such that they point to the object’s new location.

If even one object in the heap is moved, many references may need to be remapped. More importantly, each and every reference in memory must be correctly checked for potential remapping, and the remapping need must be handled safely before the program is allowed to continue executing.

With the exception of the Azul’s JVM™, virtually all commercial J2SE implementations available today perform this necessary compaction and remapping step with the program paused. Azul’s garbage collector is unique in the enterprise Java market in its ability to relocate objects and safely remap all references to them while the application execution is ongoing.

Compaction, Response Time, and Practical Heap Size

When a JVM pauses for compaction, practical Heap Size becomes directly limited.

Since pausing an application for the duration of a compaction step is highly disruptive, to the point of apparent failure, compaction tends to drive the upper-bound on the amount of practical heap that an application can utilize.

The amount of time spent in compaction tends to be linear to the size of the heap. A larger heap means longer compaction times, and on a JVM that pauses for compaction, that means longer pause times. Since most applications have some basic level of worst-case response time requirement, and these requirements coupled with inevitable compaction pauses end up dictating a limit on practical heap size.

Azul’s ability to concurrently compact the heap, and to allow the application to continue to execute while remapping is performed, allows applications to completely separate heap size from response time requirements. Applications that leverage the Azul JVM can practically make use of 10s or 100s of GB of memory without encountering compaction related pauses, simultaneously sustaining scale and consistent response times.

Fragger

Fragger is a heap fragmentation inducer, meant to induce compaction of the heap on a regular basis using a limited amount of CPU and memory resources.

The Fragger tool is to aid application testers in inducing inevitable-but-rare garbage collection events, such that they would occur on a regular and more frequent and reliable basis. Doing so allows the characterization of system behavior, such as response time envelope, within practical test cycle times.

Fragger works on the simple basis of repeatedly generating large sets of objects of a given size, pruning each set down to a much smaller remaining live set, and increasing the object size between passes such that is becomes unlikely to fit in the areas freed up by objects released in a previous pass without some amount of compaction. Fragger ages object sets before pruning them down in order to bypass potential artificial early compaction by young generation collectors.

By the time enough passes are done such that the total allocated space roughly matches the heap size (although a much smaller percentage is actually alive), some level of compaction likely becomes inevitable.

Fragger's resource consumption is completely tunable, it will throttle itself to a tunable rate of allocation, and limit it's heap footprint to configurable level.

When run with default settings, Fragger will occupy ~25% of the total heap space, and allocate objects at a rate of 20MB/sec. At these settings compaction will usually occur within 2 minutes per GB of heap. Altering the target allocation rate, as well as the heap occupancy ratio and with the number of passes in a compaction-inducing iteration, will change the frequency with which compactions occur, and the CPU percentage consumed by Fragger.

Blogs

Cliff Click's blog »

Peter Holditch's blog »

Kirk Pepperdine's blog »

Brian Goetz's blog »

AVM

1. Azul Virtual Machine product.

2. AVM process that runs on the appliance and actually contains the threads and memory of the JVM. The AVM process talks to the proxy process via TCP.

AVX

Azul system software product. Software to manage an individual appliance.

Azperfbar

Graphical performance visualization tool showing instantaneous core utilization. Accessible via the NP-shell.

AzTEK

Azul thread execution kernel. This is the custom OS that manages CPU and memory resources on an appliance. AzTEK was designed to be lightweight and scalable (current appliances contain up to 768 cores). Part of AVX.

Aztop

A text-based tool showing the top resource-consuming applications running on an appliance. Similar to Linux top. Accessible via the NP-shell.

CLI

Command-line interface for an appliance. Accessible via serial port or ssh.

Core

Hardware CPU processing element capable of executing a software thread.

CPM

Compute Pool Manager. Azul's domain management software. Consists of Management Server and Policy Server.

DirectPath

Azul's I/O technology that optimizes traffic between two AVM instances running on Azul appliances. Their corresponding proxies are bypassed reducing I/O traffic to the proxy hosts.

DM

Data Manager. A data collection tool that records data for the domain into .csv format at periodic intervals. Intended as a platform on top of which customers can write applications (to do billing, for example).

Domain

The set of appliances managed by a single CPM cloud (MSVIP and PSVIP pair).

MS

Management Server. Domain management and monitoring application. Owns MSVIP. Part of CPM.

MSVIP

Management Server virtual IP address. IP address that admin users can point a web browser to to access the domain management interface. Owned by Management Server.

Npsh

NP-shell for an appliance. Accessible via the 'gonpsh' command in the CLI. This is a restricted shell environment that provides a Linux-like feel, and various performance and diagnostic commands.

OTC

Optimistic Thread Concurrency. Transactional memory technology built into the Vega chip architecture. Allows multiple threads to simultaneously acquire the same lock and make forward progress. Rollbacks occur on true data collison conflicts to preserve critical section semantics.

Pauseless garbage collection

Azul's proprietary garbage collector handles heaps in the hundreds of gigabytes without introducing application pauses. Comes in two flavors: pauseless (original) and generational pauseless (latest). GPGC has young and old generations (like many other GC algorithms).

Pool

A collection of appliances that form a virtual resource set. A domain can have many pools. Every pool belongs to exactly one domain. Every appliance belongs to exactly one pool.

Proxy

1. Proxy host. The standard (for example) Linux host where the Azul java instance is launched from. The host where a proxy process runs.

2. Proxy process. The stub java process running on a standard (for example) Linux host. The proxy process talks to the AVM via TCP.

PS

Policy Server. Domain launch management application. Owns PSVIP. Part of CPM.

PSVIP

Policy Server virtual IP address. IP address that the proxy process talks to during launch time. Owned by Policy Server.

RTPM

REALTime Performance Monitor. A web browser-based tool for visualizing Java application behavior. RTPM is built into the Azul JVM, requires no configuration, and has no overhead.

UDC

Universal Data Collector. A tool for seamlessly collecting CPU, Memory and I/O data from an appliance, running applications, as well as their proxy hosts. This tool is helpful when you want to collect data for your proxy host as well as an appliance.

Vega

The name of the Azul chip architecture. Individual chip generations are referred to as Vega1, Vega2, etc. Vega1 chips have 24 cores, Vega2 chips have 48 cores, and Vega3 chips have 54 cores on a die.