In Auto-Tuning mode, SlotMaximizer attempts many potential optimizations on kernels directed by the developer.  Each of the many optimizations of a given kernel are executed using the actual data being processed by the application, and performance statistics of the optimized kernels processing this data are provided to the developer to select the appropriate version to leverage.

In production code the original kernel is used, and the selected configuration for the preferred optimizations is indicated to the compiler using an attribute that is provided to the developer as part of the Auto-Tuning output data.

SlotMaximizer is already incorporated into the AMD Catalyst™ drivers as a preview and can be used by anyone developing applications using the APP SDK.  It will be turned on by default to support application execution on end-user systems later in the year.

Improvements continue to be made to SlotMaximizer, download the latest beta version from   here.  You will need to register on the Multicoreware web-site, but this will also give you access to report issues, ask questions, etc. using their forums.  The download will install over that included with your AMD Catalyst™ drivers, and includes documentation on use of SlotMaximizer.

Mark Ireton is the Product Manager for Compute Solutions at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

http://developer.amd.com/cpu/codeanalyst/codeanalystlinux

Last quarter, we saw lot of active users suggesting new enhancements, posting queries and bugs. You are helping us to make AMD CodeAnalyst better. Thank you very much for your feedback.

In the AMD CodeAnalyst 3.2 release, we have added support to profile Java inline methods. To profile Java inline methods, AMD CodeAnalyst requires JDK version 1.7 or higher. We have added support for the new Linux distributions RHEL 5 Update 7 and RHEL 6 Update 2.

Multiple bugs are fixed to increase the profiling experience of Java Applications. Incorrect block analysis in Disassembly and Module data views are fixed. Added support for nested java inline functions. Various GUI crashes in Module Data View were fixed.

Up until version 3.1, AMD CodeAnalyst was using Qt version 3.3. In this release we have migrated to Qt version 4.x. This would make us to use the new features of Qt to improve the user experience.

Apart from adding new features, we are also committed to make our product more robust by fixing more bugs. If you have any suggestion to improve the tool or whenever you experience any bug, please let us know, through our forums or comments to this blog.

http://developer.amd.com/cpu/codeanalyst/codeanalystwindows

Based on many valuable suggestions and feedback we received from our end-users, we are excited to say that the following new features have been added in this release:

1.       Cache Line Utilization

2.       Data translation is made faster now with multiple threads

3.       OpenCL™ Kernel Source Code level support

4.       Improved Source & Assembly Profile display

5.       Importing standalone oclt file into project session

6.       PID/TID Filtering

7.       Improved View Management for individual cores

8.       Timeline Navigation

9.       User configurable Maximum # of run lanes

10.   AMD CodeAnalyst has been outfitted with an array  of bugs fixes.

We hope that this release helps you with your work efficiently and pleasantly. If you encounter  any bugs or see where AMD CodeAnalyst could be improved, please reach us through our forums or by replying this blog page.

Jeganathan Swaminathan is a Member of Technical  Staff at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

OpenCL and the OpenCL logo are trademarks of Apple Inc. used with permission by Khronos

Let’s start with the OpenCL 1.2 preview.    Key new features supported in the preview are Host access flags for memory objects, greater flexibility for 1D and 2D image and buffer representations, memory object migration, a new generalized image creation API and more, see list below   These preview features are currently only supported on the GPU, but during the first half of 2012 expect a complete implementation of all OpenCL required features and select extensions on both CPU and GPU.  Start making your plans now to join us at AFDS ’12 in Belleview WA this coming June to see demonstrations using the new capabilities.

OpenCL 1.2 aside, one of the new features that I am really excited about is the preview of the OpenCL Static C++ Kernel Language Extension.   This is a version of C++ that can be used to write OpenCL kernels.    “This is huge!” was a comment from one of our ISVs when I disclosed this capability during a meeting at SC’11.  Key capabilities of OpenCL Static C++ include kernel and function overloading, kernel and member templates, inheritance, friend classes, and more.  For full details on supported and unsupported C++ capabilities please read the specification for OpenCL Static C++ Kernel Language Extension

Both the OpenCL and the OpenCL Static C++ Kernel Language Extension previews are supported in a special driver release that can be downloaded from the OpenCL SDK download page.

Pressing forward our commitment to make OpenCL easier to use, with this new SDK we are including the new Khronos C++ Wrapper API.  This new API adds many benefits to the developer for host side programming in C++.  It is no longer necessary to specify platform, context, or queue OpenCL host side objects, greatly simplifying host side code, programs are automatically built on creation for all devices in the associated context, and error checking is included by default.  Finally the new make kernel function replaces the need for many OpenCL commands and helps improve type checking and overall code robustness.  All of these changes taken together mean that it is now possible to write a complete OpenCL Hello World, including host side code, in just 20 lines of C++.

This  release was designed to provide comprehensive performance improvements, both on the CPU side with key run-time performance improvements and support for both AVX and FMA4 instructions, and on the GPU side with the addition of asynchronous data copy and kernel execution (preview), support for atomic counters, and support for the cl_amd_media_ops2 OpenCL extension.

From AMD Catalyst™ software 11.11 we are now also including the OpenCL runtime in the AMD Catalyst software Linux drivers.  This ensures that the maximum number of end users are enabled for use of OpenCL applications.

Key features supported in SDK 2.6 and the AMD Catalyst™ software 11.12 drivers include:

• Inclusion of the Khronos C++ Wrapper API
• OpenCL runtime integration into Linux in addition to Windows® AMD Catalyst™ drivers.
• Multi GPU support on Linux platforms.
• PX5 support.
• Support for AVX extensions for CPUs that support this extension.
• Support for FMA4  extensions in OpenCL built-in function libraries for CPUs that support this extension
• Kernel Reflection, query kernel parameters and enable use of OpenCL kernels in data driven applications.
• Support for Atomic counters on fusion devices.
• khr_fp64 support on AMD Radeon™ HD 69xx graphics devices.
• Redesign on OpenCL run-time on CPU significantly helps improve performance.
• Support for the cl_amd_media_ops2 extension, exposing hardware capabilities for accelerating image related processing.
• Async copies preview (set environment variable GPU_ASYNC_MEM_COPY=2 to enable).

The OpenCL 1.2 preview includes the following capabilities:

• Host access flags for memory objects enables more efficient buffer handling
• Pattern based GPU buffer and image initialization eliminates need for certain buffer/image transfers
• Memory objects migration supports transfer of buffer prior to need
• New generalized image creation API
• Enhanced image/buffer map operations
• OpenCL 1.2 CPU device partition including partition of a CPU after addition to a context
• Generalized 1D and 2D images, image arrays,  and image<-> buffer interop

More details on how to access the OpenCL 1.2 preview are provided on the OpenCL SDK 2.6 developer  page.

gDEBugger version 6.1 is a major performance and robustness improvements over version 6.0 and nan be downloaded for use with this SDK from http://developer.amd.com/gDEBugger.

• Integrated with Microsoft® Visual Studio®
• Stand alone version
• Registration no longer necessary for obtaining a license

APP KernelAnalyzer v 2.0:

• Support for AMD Radeon™ HD 7000 series GPUs (compilation only – no analysis)
• Support for AMD Catalyst™ software revisions through 11.11
• Support for compiling kernels with the installed driver (select Installed Driver under the CAL version in the Options panel)
• Format and Target Object Code are now separated.

APP Profiler v2.4 includes several key new features, including:

• A kernel occupancy analyzer which estimates, for each kernel dispatch, the number of in-flight wavefronts on a compute unit as a percentage of the theoretical maximum number of wavefronts that the compute unit can support. In addition to reporting the occupancy percentage, the profiler can display a report which can help the developer to achieve a higher occupancy percentage.
• The ability to navigate from the API trace to the source code that called an OpenCL API.
• OpenCL API analysis which provides performance suggestions to the developer.
• The ability to filter which OpenCL APIs will be traced.
• Several UI enhancements, including the ability to rename sessions from the Session Explorer Window, and the ability to automatically delete profiler sessions when closing a Microsoft® Visual Studio solution®.
• Preview: Support for profiling with AMD Radeon™ HD 7000 series GPUs (requires AMD APP SDK v2.6 and an AMD Catalyst™ software version that supports this hardware).

On a final note – remember to install the AMD Catalyst™ software 12.1 drivers when they are released in January – there are significant new capabilities that are going to be made available then.  But more on that later.

Mark Ireton is the Product Manager for Compute Solutions at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

As you may be well aware, here at AMD we are continuing on our commitment of providing rich and comprehensive set of tools to help developers extract the benefits of heterogeneous computing platforms. We firmly believe industry standards, such as OpenCL, help achieve the true freedom of choice in GPU compute platforms. In order to drive further adoption of these standards, it is critical that we enable our software ecosystem partners including the tool vendors such as MulticoreWare Inc. to make sure they fully support OpenCL. We all share a common set of goals – i.e. help developers extract the maximum performance and achieve overall productivity on heterogeneous compute platforms.

  Here is the short summary for each of MCW tool:

1. Global Memory for Accelerators (GMAC) enables development of code for CPUs and GPUs in a single pointer space, without requiring the developer to be aware of, or manage, the different memory domains and allocations.
2. Task Manager™ ensures effective sequencing of operations across both CPUs and GPUs, with resulting optimization of memory accesses, vector operations and more.
3. Parallel Path Analyzer (PPA) provides an interactive drill-down and Gantt-chart view of the execution time of various OpenCL kernels within a specific code-base. This can be used to optimize kernels within the critical path to achieve better performance.

These are beta tools so we need your help to make sure they stand up to real world use.  We are working on a reward system for developers who give valuable feedback.  Details will be coming soon! 

How to participate:

1. Register at www.multicorewareinc.com
2. Download and use the tool(s)
3. Share your comments, ask questions, or report issues in the interactive user forum

By the way, did I mention these tools are free for  use on AMD platforms , just like all of the AMD tools. Look forward to hearing from you.

-Milind Kukanur

Milind Kukanur is the Product Manager for Software Tools at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

You will have seen the recent announcement of the ratification of the OpenCL 1.2 standard, to which AMD was one of the key contributors.  Demonstrating our commitment to OpenCL, SDK 2.6 (available on December 13) will be previewing core features of OpenCL 1.2.  OpenCL continues to increase in popularity as I discussed in my prior blog, and just this week Altera has also announced support for OpenCL.  During the first half of next year we will be rolling out a complete implementation of OpenCL 1.2 for both CPU and GPU and demonstrating the latest capabilities.

As you are probably already aware, many of the key features of OpenCL are incorporated into the OpenCL run-time, which since last summer has been delivered on Windows® with the monthly Catalyst driver releases as well as with the SDK.  As of the Catalyst 11.11 driver release this month, the OpenCL run-time for Linux is now also included in the Linux Catalyst drivers.  This heralds an important change in how, and how often, we can provide updates for OpenCL.  As we move into 2012, as well as providing two to three SDK releases a year, where we will provide updated samples, API’s documentation etc., we will also be upgrading our OpenCL solution on a more frequent basis through the regular monthly Catalyst driver updates.  Look for a new section in the Catalyst driver release notes specific to OpenCL capabilities.  Including the OpenCL run-time in the Catalyst drivers is a key component in ensuring that the millions of PC’s out there that include AMD GPU technology that can be leveraged by your OpenCL enabled application.

Of course there are many new enhancements that that will be available with SDK 2.6 and Catalyst 11.12.  Join me here again in mid-December to read more.

Mark Ireton is the Product Manager for Compute Solutions at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

Let me reflect for a moment on the factors that I believe have led to this. OpenCL has always been an open standard driven by key semiconductor and software companies, but this in itself is not enough.  No, in my mind, the real change is that the promise of multi-vendor cross-platform support is now a tangible reality.  AMD, Intel, and Nvidia already have OpenCL SDKs publically available, with AMD’s SDK supporting both GPU and X86 CPU. Furthermore, “…ARM is committed to OpenCL…..With our hardware architecture, and (just as importantly) our software architecture, we are supporting all the GPU computing APIs that are needed by the industry to take best advantage of heterogeneous computing” according to Jem Davies¹.  Software vendors are seeing the opportunity — already over 50 leading applications are accelerated by AMD Fusion APUs².

The other key element, of course, is the increasing availability of amazing compute capability in mainstream products. A few weeks ago my wife bought a new laptop containing an AMD A6-3400M APU with AMD Radeon™ HD graphics.  This platform is currently listed by HP as “Starting from $599”³.  AMD A-Series APUs can deliver supercomputer-like performance of more than 500 GFLOPS of raw compute performance⁴ and data transfer rates between the CPU and the GPU can be as high as 15GB per second⁵. But perhaps more importantly, with the exception of the desktop enthusiast space AMD’s client roadmap is entirely based on APUs.  AMD had already shipped over 12 million APUs⁶ as of August of this year, and we’re still counting!

Here’s to a strong future for OpenCL!

¹Jem Davies, Fellow & VP of Technology in the Media Processing Division of ARM

http://blogs.arm.com/multimedia/515-arm-discusses-future-in-keynote-heterogeneous-computing-cpus-and-gpus

²http://www.amd.com/us/press-releases/Pages/amd-fusion-apus-acc-mr-50-app-2011mar07.aspx

³http://www.shopping.hp.com, October 21^st 2011.

⁴http://www.amd.com/us/press-releases/Pages/amd-a-series-desktop-2011june30.aspx

⁵A8-3800 with Radeon™ HD 6550D graphics, 8GB DDR3-1333.  http://www.amd.com/us/press-releases/Pages/app-sdk-2011aug08.aspx

⁶http://www.amd.com/us/press-releases/Pages/amd-boosts-fusion-apus-2011aug22.aspx

Mark Ireton is the Product Manager for Compute Solutions at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites, and references to third party trademarks, are provided for convenience and illustrative purposes only. Unless explicitly stated, AMD is not responsible for the contents of such links, and no third party endorsement of AMD or any of its products is implied.

http://developer.amd.com/cpu/codeanalyst/codeanalystlinux

Last quarter, a lot of active users suggested new enhancements, posted queries and bugs. You are helping us to make AMD CodeAnalyst software better. Thank you very much for your feedback.

In this AMD CodeAnalyst 3.1 release, we have enabled public support for Comal Platforms. Now the public users can profile their applications on “Trinity” processor-based “Comal” platforms. We have added support for the new Linux distribution Ubuntu-11.10.

In the previous release, we redesigned the data navigation tabs. In this release, we have made some enhancements and fixed quite a few bugs in the “Source” view tab, all designed to improve the user experience while analyzing the profile data.

We have also added support to profile the applications compiled by Open64 compilers suite. This can help users to make use of AMD CodeAnalyst to profile and analyze the performance characteristics of the applications compiled using Open64 compilers.

Apart from adding new features, we are also committed to making our product highly robust by fixing bugs. If you have any suggestion to improve the tool or if you experience any bug, please let us know, through our forums or comments to this blog.

What is NUMA?

Many multiprocessor systems today, including those designed with AMD processors, use a Non-Uniform Memory Access (NUMA) memory design. In a NUMA design, the total set of processor cores is divided into NUMA nodes. Each NUMA node and has its own local memory controller with its own local memory and the processors in a node can also access in a cache coherent manner the memory owned by the other NUMA nodes. All the memory appears in a single address space. Generally the time to access local node memory is less than that to access remote node memory, hence the name NUMA.

 Now consider an application that has enough software threads that it will use the cores from many (or all) NUMA nodes. The following general guidelines for memory allocation can help performance in a NUMA system:

1. Memory regions that are only accessed by the cores in one node should be allocated from that node.
2. Memory regions that are accessed globally by cores on many nodes should be interleaved across all those nodes.

The first guideline seems obvious since the access time to local memory is less. However in many NUMA designs there is not a large difference in remote vs. local access time so this can be less critical.

A more important performance consideration can be to avoid having all the cores competing for the memory on one node. The memory controller on the heavily accessed node can be a bottleneck. Both guidelines above guard against this condition.

NUMA and the Java Heap

All Java Objects are allocated from the heap and are subject to being moved during garbage collection. How is the Java Heap allocated by the JVM? The heap generally consists of contiguous virtual address space but how it is mapped to physical memory depends on the OS’s underlying allocation policy and whether the JVM does anything to override that policy.

The default allocation policies for Linux® and Windows OSs are quite different.

• Linux uses a “first touch” policy which means it will try to allocate on the node on which the core that first touched the page resides.
• Windows uses a “process home node” policy. Each process is assigned a home node at random. The OS will try to allocate memory on that home node first, even if the core that is touching the memory is running on some other node.

The thread allocation policy should also be noted here and is similar on both OSes:

• The first thread will be scheduled on the least busy node (the “home node” on Windows) and subsequent threads will also try to schedule on that same node until that node is overloaded and other nodes are idle.

From the above, we see that on Linux the way the Java Heap gets allocated depends on the way the threads start up and touch memory, and on the total size of the heap.

• If a large number of threads (enough to be scheduled across several NUMA nodes) start up early and start touching memory, the heap will tend to be interleaved across the NUMA nodes from the multiple threads touching memory.
• If a small number of threads (or by limit one thread) starts up and touch enough memory before the larger number of threads spin up, the heap will tend to be allocated on one node first, then overflow to the next node. If the total size of the heap is close to the total amount of physical memory, the heap will tend to be spread across nodes, otherwise, it will tend to be heavily weighted to the startup nodes. In addition, even if the heap is spread across nodes, very large ranges of virtual address space will be mapped to the same node, which can be very different from the recommendation that memory be interleaved across the nodes.

On Windows, we will always get the second bullet above, because of the “process home node” allocation policy.

Java Deployment Workarounds

Are there things that can be done at Java deployment time to alleviate in particular the bottleneck of cores from many NUMA nodes trying to access memory on one NUMA node?

• We mentioned above that for a heap size that is closer to the total amount of physical memory, the heap must by definition be spread across the nodes, although the interleaving may be poor and large ranges of virtual address space can be mapped to the same node. Thus this by itself is not an ideal solution.
• The allocation problems occur when the application has enough software threads that it will use the cores from many (or all) NUMA nodes. Sometimes it is possible to split an application into several processes, each one having a smaller number of threads such that it will tend to use the cores from one NUMA node and to pick a heap that is small enough to fit on one NUMA node. In those cases, the OS will tend to allocate the threads and memory on that one node. In a more extreme step, the OS can be effectively taken out of the picture by affinitizing the processes manually to NUMA nodes using “numactl” on Linux or “start /affinity” on windows. You may have noticed several benchmark submissions taking this strategy.

However, in many cases it is not possible to split an application into several processes, so you do need to run many threads in one process and you want a heap size that happens to fit on one or two NUMA nodes. In those cases, you’d like the JVM to help you achieve the allocation guidelines.

JVM Memory Allocation Overrides

The Hotspot JVM added an option a few years ago called UseNUMA. Internally this option divides the heap into two types of chunks, numa_local and numa_global. A numa_local chunk is local to one node, while a numa_global chunk is interleaved across all nodes. These types, when used wisely by the garbage collector, can help meet the allocation guidelines:

• Memory that could be accessed by many nodes such as the areas that have survived GC collections will be numa_global or interleaved.
• A Thread Local allocation buffer used for new allocations by a thread will be selected from a pool of chunks which are numa_local to that node.

Note that this policy treats objects that survive collections as global (interleaved), whereas some of these may in fact still be accessed by only one node. Still, the UseNUMA policy can be a big improvement over just using the default OS allocation policy for the heap.

The internal design requirements for numa_local and numa_global required that chunks be able to be resized dynamically and possibly have their type changed from local to global as the heap grew over time. While this meshed well with the memory allocation APIs of Linux and Solaris, unfortunately, it was not possible to implement this full UseNUMA design on Windows. So the UseNUMA option was effectively a no-op on Windows.

Extending a subset of UseNUMA to Windows

Here at AMD we noticed that while it was not possible to implement the full UseNUMA functionality on Windows, one could implement the numa_global or interleaved functionality. In this strategy, all parts of the heap would be interleaved across the NUMA nodes. The Windows API call VirtualAllocExNuma can be used to allocate memory on a specific node to achieve this interleaving.

We noticed that implementing just this numa_global interleaving subset could still result in some very good performance improvements on those applications where you do need to run many heap-accessing threads in one java process especially when you have a heap size that happens to fit on say one or two NUMA nodes. We saw up to 2.5x improvement on some standard benchmarks under these conditions. (Note that an application that runs few enough threads that they would map to a single NUMA node should already be getting good memory usage from the default Windows policy and so would not benefit from this memory interleaving)..

We submitted this functionality to the OpenJDK under a new option called UseNUMAInterleaving. This option is available on Windows, Linux and Solaris, although in most cases on Linux and Solaris you would just use the full UseNUMA option. In addition, on Windows, for now the UseNUMA option will map to UseNUMAInterleaving, since this subset is the only NUMA functionality available there. (It is possible in the future that a full UseNUMA would be implemented for Windows).

This new option is available in the early access releases of JDK7 Update 2, starting with build 08. So if you’re running a lot of threads in a single JVM, give it a try on your Windows Java deployment. We are interested in hearing of your experiences with this new option.

http://blogs.amd.com/developer/2011/07/12/apache-hadoop-performance-tuning-methodologies-and-best-practices-%E2%80%93-a-downloadable-reference-guide/

This week I made a pleasant discovery that would be easily overlooked by almost everyone, so I wanted to point it out for Hadoop users. One of my Hadoop test beds is a one machine pseudo-cluster with one AMD OpteronTM™ 6128 processor and 5 hard drives. This system happens to have Ubuntu 10.10, Hadoop 0.20.3 and JDK 6u27 installed. My test case here is 16GB terasort.

As you may know, lzo compression (see http://www.oberhumer.com/opensource/lzo/ ) is commonly used in Hadoop for compressing the map output, for example, see http://wiki.apache.org/hadoop/UsingLzoCompression. Compressing the map output can help improve the performance by decreasing the amount of writes to the disks, which should help reduce I/O wait time depending on your disk configuration.

I happened to download the latest lzo source package v2.06 and built it with the default options with the default gcc 4.4.5 in Ubuntu 10.10. This is a small library and it gets compiled in under a minute with parallel compilation. As usual it gets installed in /usr/local/lib. It turned out that the terasort run is noticeably faster with v2.06 than the Ubuntu 10.10 default lzo which is v2.03.

Percent improvement with v2.06 over default v2.03

Nothing else was changed in the environment. The execution time got about 5% faster with the latest v2.06 compared to the default lzo library, with low run-to-run variance. I checked another testbed which is running Fedora 14, and it turns out Fedora 14 also has v2.03 by default. So I recommend checking the lzo version regardless of your linux distro.

Of course your workload is different from terasort, and the benefit of this change depends on the size of your map output. But overall that is a nice 5% speedup in this workload for a couple minutes’ work downloading, building, and installing a newer version of an open source library that you are probably already using.