openRuyi-Project/gcc
3
0
Fork 0
mirror of https://github.com/openRuyi-Project/gcc.git synced 2026-09-05 13:31:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
227,552 Commits 2 Branches 0 Tags
master
Commit Graph
80 Commits
Author SHA1 Message Date
Andrew Stubbs a53c5c42c6 libgomp, amdgcn, nvptx: Improve omp_target_is_accessible [PR121813] 
This patch extends omp_target_is_accessible to check the actual device status
for the memory region, on amdgcn and nvptx devices (rather than just checking
if shared memory is enabled).

In both cases, we check the status of each 4k region within the given memory
range (assuming 4k pages should be safe for all the currently supported hosts)
and returns true if all of the pages report accessible.

The testcases have been modified to check that allocations marked accessible
actually are accessible (inaccessibility can't be checked without invoking
memory faults), and to understand that some parts of an array can be accessible
but other parts not (I have observed this intermittently for the stack memory
on amdgcn using the Fortran testcase, which can have the allocation span pages).

There's also new testcases for the various other memory modes, and for managed
memory.

include/ChangeLog:

	* cuda/cuda.h (CUpointer_attribute): New enum.
	(cuPointerGetAttribute): New prototype.

libgomp/ChangeLog:

	PR libgomp/121813
	PR libgomp/113213
	* libgomp-plugin.h (GOMP_OFFLOAD_is_accessible_ptr): New prototype.
	* libgomp.h
	(struct gomp_device_descr): Add GOMP_OFFLOAD_is_accessible_ptr.
	* libgomp.texi: Update omp_target_is_accessible docs.
	* plugin/cuda-lib.def (cuPointerGetAttribute): New entry.
	* plugin/plugin-gcn.c (struct hsa_runtime_fn_info): Add
	hsa_amd_svm_attributes_get_fn and hsa_amd_pointer_info_fn.
	(init_hsa_runtime_functions): Add hsa_amd_svm_attributes_get and
	hsa_amd_pointer_info.
	(enum accessible): New enum type.
	(host_memory_is_accessible): New function.
	(device_memory_is_accessible): New function.
	(GOMP_OFFLOAD_is_accessible_ptr): New function.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_is_accessible_ptr): Likewise.
	* target.c (omp_target_is_accessible): Call is_accessible_ptr_func.
	(gomp_load_plugin_for_device): Add is_accessible_ptr.
	* testsuite/libgomp.c-c++-common/target-is-accessible-1.c: Rework
	to match more details of the GPU implementation.
	* testsuite/libgomp.fortran/target-is-accessible-1.f90: Likewise.
	* testsuite/libgomp.c-c++-common/target-is-accessible-2.c: New test.
	* testsuite/libgomp.c-c++-common/target-is-accessible-3.c: New test.
	* testsuite/libgomp.c-c++-common/target-is-accessible-4.c: New test.
	* testsuite/libgomp.c-c++-common/target-is-accessible-5.c: New test.
 2026-02-03 13:17:38 +00:00
Jakub Jelinek 254a858ae7 Update copyright years. 2026-01-02 09:56:11 +01:00
Andrew Stubbs 1d1d12da6d amdgcn, libgomp: improve generic device errors 
Switching to use "generic" ISA variants has changed the error modes a bit.

This patch changes the runtime so that it doesn't say to use the
device-specific -march option when the real problem is not the ISA (it'll be a
mismatched xnack setting, probably).

Additionally, the testsuite effective target check needs to see if the xnack
mode is accepted by the runtime, as well as the compiler.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (generic_isa_code): New function.
	(isa_matches_agent): Use generic ISA details to help select an error
	message on ISA mismatch.
	* testsuite/lib/libgomp.exp
	(check_effective_target_offload_target_amdgcn_with_xnack): Use a
	runtime check.
 2025-12-04 15:35:05 +00:00
Andrew Stubbs 723b18ce3d libgomp, amdgcn: Implement Managed Memory 
This patch implements "managed" memory for AMD GCN GPUs in OpenMP.  It
builds on the support added to the NVPTX libgomp for CUDA Managed Memory, a
week or two ago.

These features were first posted here a few years ago, as part of a larger
Unified Shared Memory patch series, and then in a slightly changed version just
over a year ago.  Hopefully this time the controversial bits have been removed.

Since we do not use HIP we cannot use hipMallocManaged, so this patch attempts
to replicate the same effect by setting the appropriate attributes.  This works
on more devices than support proper USM, but still I cannot be sure that the
settings are correct for every device out there (I have tested on gfx900,
gfx906, gfx908, gfx90a, and gfx1100).

The HSA header file update uses the most recent files relicensed for us by AMD,
at the time of the first patch posting.  Those files have certainly moved on in
the upstream sources, but I did not ask to get those relicensed.

include/ChangeLog:

	* hsa.h: Import newer version.
	* hsa_ext_amd.h: Likewise.
	* hsa_ext_image.h: Likewise.

libgomp/ChangeLog:

	* Makefile.in: Regenerate.
	* libgomp-plugin.h (gomp_simple_alloc_init_context): New prototype.
	(gomp_simple_alloc_register_memory): New prototype.
	(gomp_simple_alloc): New prototype.
	(gomp_simple_free): New prototype.
	(gomp_simple_realloc): New prototype.
	* libgomp.h (gomp_simple_alloc_init_context): Move to libgomp-plugin.h.
	(gomp_simple_alloc_register_memory): Likewise.
	(gomp_simple_alloc): Likewise.
	(gomp_simple_free): Likewise.
	(gomp_simple_realloc): Likewise.
	* libgomp.texi: Update AMD managed memory description.
	* plugin/Makefrag.am (libgomp_plugin_gcn_la_SOURCES): Add
	simple-allocator.c and plugin/mutex.c.
	* plugin/plugin-gcn.c: Include sys/mman.h and unistd.h.
	(struct hsa_runtime_fn_info): Add hsa_amd_svm_attributes_set_fn.
	(dump_hsa_system_info): Add HSA_AMD_SYSTEM_INFO_SVM_SUPPORTED and
	HSA_AMD_SYSTEM_INFO_SVM_ACCESSIBLE_BY_DEFAULT to the GCN_DEBUG output.
	(init_hsa_runtime_functions): Add hsa_amd_svm_attributes_set.
	(isa_matches_agent): Add a new error message for the case where the
	ISA doesn't match but the name does.
	(managed_ctx): New variable.
	(managed_heap_create): New function.
	(GOMP_OFFLOAD_managed_alloc): Likewise.
	(GOMP_OFFLOAD_managed_free): Likewise.
	* simple-allocator.c (gomp_fatal): New macro.
	* testsuite/lib/libgomp.exp (check_effective_target_omp_managedmem):
	Add amdgcn support checker.
	(check_effective_target_offload_target_amdgcn_with_xnack): New.
	* testsuite/libgomp.c-c++-common/requires-4.c: Ignore xnack warning.
	* testsuite/libgomp.c-c++-common/requires-4a.c: Ignore xnack warning.
	* testsuite/libgomp.c-c++-common/requires-5.c: Ignore xnack warning.
	* testsuite/libgomp.c++/alloc-managed-1.C: Add -mxnack=on, if needed.
	* testsuite/libgomp.c/alloc-managed-1.c: Likewise.
	* testsuite/libgomp.c/alloc-managed-2.c: Likewise.
	* testsuite/libgomp.c/alloc-managed-3.c: Likewise.
	* testsuite/libgomp.c/alloc-managed-4.c: Likewise.
	* testsuite/libgomp.fortran/alloc-managed-1.f90: Likewise.
	* plugin/mutex.c: New file.
 2025-12-01 12:03:35 +00:00
Tobias Burnus d2ad7e9083 libgomp: Add is_integrated_apu function to plugin/plugin-{gcn,nvptx}.c 
The added function is currently '#if 0' but is planned to be used to enable
self mapping automatically. Prerequisite for auto self maps is still mapping
'declare target' variables (if any, in libgomp) or converting all
'declare target' variables to 'declare target link' in the compiler
(as required for 'omp requires self_maps').

include/ChangeLog:

	* hsa_ext_amd.h (enum hsa_amd_agent_info_s): Add
	HSA_AMD_AGENT_INFO_MEMORY_PROPERTIES.
	(enum): Add HSA_AMD_MEMORY_PROPERTY_AGENT_IS_APU.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (is_integrated_apu): New; currently '#if 0'.
	* plugin/plugin-nvptx.c (is_integrated_apu): Likewise.
 2025-10-10 09:48:37 +02:00
Tobias Burnus da5803c794 libgomp: Init hash table for 'indirect'-clause of 'declare target' on the host [PR114445, PR119857] 
Especially with unified-shared memory and especially with C++'s virtual
functions, it is not uncommon to have on the device a function pointer
that points to the host function - but has an associated device.
If the pointed-to function is (explicitly or implicitly) 'declare target'
with the 'indirect' clause, it is added to the lookup table.

Before this commit, the conversion of the lookup table into a lookup
hash table happened every time a device kernel was launched on the first
team - albeit if already converted, the function immediately returned.

Ignoring the overhead, there was also a race: If multiple teams were
launched, it could happen that another team of the same target region
already tried to use the lookup table which it was still being created.
Likewise when lauching a kernel with 'nowait' and directly afterward
another kernel, there could be a race of creating the table.

With this commit, the creating of the kernel has been moved to the
host-plugin's GOMP_OFFLOAD_load_image. The previous code stored a
pointer to the host/device pointer array, which makes it hard when
creating the hash table on the host (data is needed for finding the
slot) - but accessing it on the device (where the lookup has to work
as well). As the hash-table implementation (only) supports integral
value as payload (0 and 1 having special meaning), the solution was
to move to an uint128_t variable to store both the host and device
address.

As the host-side library is typically dynamically linked and the
device-side one statically, there is the problem of backward
compatibility. The current implementation permits both older
binaries and newer libgomp and newer binaries with older libgomp.
I could imagine us breaking the latter eventually, but for now
there is up and downward compatibility. (Obviously, the race is
only fixed if new + new is combined.)

Code wise, on the device exist GOMP_INDIRECT_ADDR_MAP which was
updated to point to the host/device-address array. Now additionally
GOMP_INDIRECT_ADDR_HMAP exists, which contains the hash-table map.

If the latter exists, libgomp only updates it and the former remains
a NULL pointer; it is also untouched if there are no indirect functions.
Being NULL therefore avoids the call to the device-side build_indirect_map.
The code also currently supports to have no hash and a linear walk. I think
that remained from testing; due to the backward-compat feature, it can
actually be turned of on either side.

libgomp/ChangeLog:

	PR libgomp/119857
	PR libgomp/114445
	* config/accel/target-indirect.c: Change to use uint128_t instead
	of a struct as data structure and add GOMP_INDIRECT_ADDR_HMAP as
	host-accessible variable.
	(struct indirect_map_t): Remove.
	(USE_HASHTAB_LOOKUP, INDIRECT_DEV_ADDR, INDIRECT_HOST_ADDR,
	SET_INDIRECT_HOST_ADDR, SET_INDIRECT_ADDRS): Define.
	(htab_free): Use __builtin_unreachable.
	(htab_hash, htab_eq, GOMP_target_map_indirect_ptr,
	build_indirect_map): Update for new representation and new
	pointer-to-hash variable.
	* config/gcn/team.c (gomp_gcn_enter_kernel): Only call
	build_indirect_map when GOMP_INDIRECT_ADDR_MAP.
	* config/nvptx/team.c (gomp_nvptx_main): Likewise.
	* libgomp-plugin.h (GOMP_INDIRECT_ADDR_HMAP): Define.
	* plugin/plugin-gcn.c: Conditionally include
	build-target-indirect-htab.h.
	(USE_HASHTAB_LOOKUP_FOR_INDIRECT): Define.
	(create_target_indirect_map): New prototype.
	(GOMP_OFFLOAD_load_image): Update to create the device's
	indirect-function hash table on the host.
	* plugin/plugin-nvptx.c: Conditionally include
	build-target-indirect-htab.h.
	(USE_HASHTAB_LOOKUP_FOR_INDIRECT): Define.
	(create_target_indirect_map): New prototype.
	(GOMP_OFFLOAD_load_image): Update to create the device's
	indirect-function hash table on the host.
	* plugin/build-target-indirect-htab.h: New file.
 2025-09-17 08:47:36 +02:00
Tobias Burnus 4e47e2f833 libgomp: Add OpenMP's omp_target_memset/omp_target_memset_async 
PR libgomp/120444

include/ChangeLog:

	* cuda/cuda.h (cuMemsetD8, cuMemsetD8Async): Declare.

libgomp/ChangeLog:

	* libgomp-plugin.h (GOMP_OFFLOAD_memset): Declare.
	* libgomp.h (struct gomp_device_descr): Add memset_func.
	* libgomp.map (GOMP_6.0.1): Add omp_target_memset{,_async}.
	* libgomp.texi (Device Memory Routines): Document them.
	* omp.h.in (omp_target_memset, omp_target_memset_async): Declare.
	* omp_lib.f90.in (omp_target_memset, omp_target_memset_async):
	Add interfaces.
	* omp_lib.h.in (omp_target_memset, omp_target_memset_async): Likewise.
	* plugin/cuda-lib.def: Add cuMemsetD8.
	* plugin/plugin-gcn.c (struct hsa_runtime_fn_info): Add
	hsa_amd_memory_fill_fn.
	(init_hsa_runtime_functions): DLSYM_OPT_FN load it.
	(GOMP_OFFLOAD_memset): New.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_memset): New.
	* target.c (omp_target_memset_int, omp_target_memset,
	omp_target_memset_async_helper, omp_target_memset_async): New.
	(gomp_load_plugin_for_device): Add DLSYM (memset).
	* testsuite/libgomp.c-c++-common/omp_target_memset.c: New test.
	* testsuite/libgomp.c-c++-common/omp_target_memset-2.c: New test.
	* testsuite/libgomp.c-c++-common/omp_target_memset-3.c: New test.
	* testsuite/libgomp.fortran/omp_target_memset.f90: New test.
	* testsuite/libgomp.fortran/omp_target_memset-2.f90: New test.
 2025-06-02 17:43:57 +02:00
Tobias Burnus f4aa6b5a8d libgomp: Add OpenACC's acc_memcpy_device{,_async} routines [PR93226] 
libgomp/ChangeLog:

	PR libgomp/93226
	* libgomp-plugin.h (GOMP_OFFLOAD_openacc_async_dev2dev): New
	prototype.
	* libgomp.h (struct acc_dispatch_t): Add dev2dev_func.
	(gomp_copy_dev2dev): New prototype.
	* libgomp.map (OACC_2.6.1): New; add acc_memcpy_device{,_async}.
	* libgomp.texi (acc_memcpy_device): New.
	* oacc-mem.c (memcpy_tofrom_device): Change to take from/to
	device boolean; use memcpy not memmove; add early return if
	size == 0 or same device + same ptr.
	(acc_memcpy_to_device, acc_memcpy_to_device_async,
	acc_memcpy_from_device, acc_memcpy_from_device_async): Update.
	(acc_memcpy_device, acc_memcpy_device_async): New.
	* openacc.f90 (acc_memcpy_device, acc_memcpy_device_async):
	Add interface.
	* openacc_lib.h (acc_memcpy_device, acc_memcpy_device_async):
	Likewise.
	* openacc.h (acc_memcpy_device, acc_memcpy_device_async): Add
	prototype.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_openacc_async_host2dev):
	Update comment.
	(GOMP_OFFLOAD_openacc_async_dev2host): Update call.
	(GOMP_OFFLOAD_openacc_async_dev2dev): New.
	* plugin/plugin-nvptx.c (cuda_memcpy_dev_sanity_check): New.
	(GOMP_OFFLOAD_dev2dev): Call it.
	(GOMP_OFFLOAD_openacc_async_dev2dev): New.
	* target.c (gomp_copy_dev2dev): New.
	(gomp_load_plugin_for_device): Load dev2dev and async_dev2dev.
	* testsuite/libgomp.oacc-c-c++-common/acc_memcpy_device-1.c: New test.
	* testsuite/libgomp.oacc-fortran/acc_memcpy_device-1.f90: New test.
 2025-05-29 22:47:06 +02:00
Tobias Burnus 41b9c3b848 libgomp/plugin: Add initial interop support to nvptx + gcn 
The interop directive operates on an opaque object that represents a
foreign runtime. This commit adds support for
this to the two offloading plugins.

For nvptx, it supports cuda, cuda_driver and hip; the latter is AMD's
version of CUDA which for Nvidia devices boils down to normal CUDA.
Thus, at the end for this limited use, cuda/cuda_driver/hip are all
the same - and for plugin-nvptx.c, the they differ only in terms of
what gets fr_id, fr_name and get_interop_type_desc return.

For gcn, it supports hip and hsa.

Regarding get-mapped-ptr-1.c: That's actually a fix for the
GOMP_interop commit r15-8654-g99e2906ae255fc that added
GOMP_DEVICE_DEFAULT_OMP_61 alias omp_default_device, which is
a conforming device number. But that test used -5 as check for a
non-conforming device number.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (_LIBGOMP_PLUGIN_INCLUDE): Define.
	(struct hsa_runtime_fn_info): Add two queue functions.
	(hipError_t, hipCtx_t, hipStream_s, hipStream_t): New types.
	(struct hip_runtime_fn_info): New.
	(hip_runtime_lib, hip_fns): New global vars.
	(init_environment_variables): Handle hip_runtime_lib.
	(init_hsa_runtime_functions): Load the two queue functions.
	(init_hip_runtime_functions, GOMP_OFFLOAD_interop,
	GOMP_OFFLOAD_get_interop_int, GOMP_OFFLOAD_get_interop_ptr,
	GOMP_OFFLOAD_get_interop_str,
	GOMP_OFFLOAD_get_interop_type_desc): New.
	* plugin/plugin-nvptx.c (_LIBGOMP_PLUGIN_INCLUDE): Define.
	(GOMP_OFFLOAD_interop, GOMP_OFFLOAD_get_interop_int,
	GOMP_OFFLOAD_get_interop_ptr, GOMP_OFFLOAD_get_interop_str,
	GOMP_OFFLOAD_get_interop_type_desc): New.
	* testsuite/libgomp.c/interop-fr-1.c: New test.
	* testsuite/libgomp.c-c++-common/get-mapped-ptr-1.c: Use -6
	not -5 as non-conforming device number.
 2025-03-21 21:39:42 +01:00
Tobias Burnus 8561e4e290 [GCN] Handle generic ISA names in libgomp's plugin-gcn.c 
libgomp/ChangeLog:

	* plugin/plugin-gcn.c (ELFABIVERSION_AMDGPU_HSA_V6,
	EF_AMDGPU_GENERIC_VERSION_V, EF_AMDGPU_GENERIC_VERSION_OFFSET,
	GET_GENERIC_VERSION): New #define.
	(elf_gcn_isa_is_generic): New.
	(isa_matches_agent): Accept all generic code objects on the first
	go; extend the diagnostic and handle runtime-failed case.
	(create_and_finalize_hsa_program): Call it also after loading
	the code failed, pass the status.
 2025-02-07 13:20:25 +01:00
Jakub Jelinek 6441eb6dc0 Update copyright years. 2025-01-02 11:59:57 +01:00
Tobias Burnus 7a12dc695b plugin/plugin-gcn.c: Fix error handling of GOMP_OFFLOAD_openacc_async_construct 
Follow up to r15-5392-g884637b6362391. As the name implies,
GOMP_OFFLOAD_openacc_async_construct is also externally called.
Hence, partially revert previous commit to permit unlocking handling
in oacc-async.c's lookup_goacc_asyncqueue by not failing fatally.

Hence, also the other (indirect) callers had to be updated:
GOMP_OFFLOAD_dev2dev fails now with 'false' and
GOMP_OFFLOAD_async_run fatally.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_dev2dev, GOMP_OFFLOAD_async_run):
	Handle omp_async_queue == NULL after call to maybe_init_omp_async.
	(GOMP_OFFLOAD_openacc_async_construct): Use error not fatal error,
	partially reverting r15-5392.
 2024-12-10 16:16:04 +01:00
Tobias Burnus 884637b636 libgomp/plugin/plugin-gcn.c: async-queue init - fix function-return type and fail fatally 
libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_openacc_async_construct): In
	case of an error, call GOMP_PLUGIN_fatal not ..._error; use NULL
	not false in return.
 2024-11-18 14:58:21 +01:00
Tobias Burnus 8473010807 libgomp/plugin/plugin-gcn.c: Show device number in ISA error message 
libgomp/ChangeLog:

	* plugin/plugin-gcn.c (isa_matches_agent): Mention the device number
	and ROCR_VISIBLE_DEVICES when reporting an ISA mismatch error.
 2024-11-11 12:17:42 +01:00
Andrew StubbsandTobias Burnus a6b26e5ea0 amdgcn: Refactor device settings into a def file 
Almost all device-specific settings are now centralised into gcn-devices.def
for the compiler, mkoffload, and libgomp.  No longer will we have to touch 10
files in multiple places just to add another device without any exotic
features.  (New ISAs and devices with incompatible metadata will continue to
need a bit more.)

In order to remove the device-specific conditionals in the code a new value
HSACO_ATTR_UNSUPPORTED has been added, indicating that the assembler will
reject any setting of that option.

This incorporates some of Tobias's patch from March 2024.

Co-Authored-By: Tobias Burnus <tburnus@baylibre.com>

gcc/ChangeLog:

	* config.gcc (amdgcn): Add gcn-device-macros.h to tm_file.
	Add gcn-tables.opt to extra_options.
	* config/gcn/gcn-hsa.h (NO_XNACK): Delete.
	(NO_SRAM_ECC): Delete.
	(SRAMOPT): Move definition to generated file gcn-device-macros.h.
	(XNACKOPT): Likewise.
	(ASM_SPEC): Redefine using generated values from gcn-device-macros.h.
	* config/gcn/gcn-opts.h
	(enum processor_type): Generate from gcn-devices.def.
	(TARGET_VEGA10): Delete.
	(TARGET_VEGA20): Delete.
	(TARGET_GFX908): Delete.
	(TARGET_GFX90a): Delete.
	(TARGET_GFX90c): Delete.
	(TARGET_GFX1030): Delete.
	(TARGET_GFX1036): Delete.
	(TARGET_GFX1100): Delete.
	(TARGET_GFX1103): Delete.
	(TARGET_XNACK): Redefine to allow for HSACO_ATTR_UNSUPPORTED.
	(enum hsaco_attr_type): Add HSACO_ATTR_UNSUPPORTED.
	(TARGET_TGSPLIT): New define.
	* config/gcn/gcn.cc (gcn_devices): New constant table.
	(gcn_option_override): Rework to use gcn_devices table.
	(gcn_omp_device_kind_arch_isa): Likewise.
	(output_file_start): Likewise.
	(gcn_hsa_declare_function_name): Rework using TARGET_* macros.
	* config/gcn/gcn.h (gcn_devices): Declare struct and table.
	(TARGET_CPU_CPP_BUILTINS): Rework using gcn_devices.
	* config/gcn/gcn.opt: Move enum data to generated file gcn-tables.opt.
	Use new names for the default values.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX900): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX906): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX908): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX90a): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX90c): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX1030): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX1036): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX1100): Delete.
	(EF_AMDGPU_MACH_AMDGCN_GFX1103): Delete.
	(enum elf_arch_code): Define using gcn-devices.def.
	(get_arch): Rework using gcn-devices.def.
	(main): Rework using gcn-devices.def
	* config/gcn/t-gcn-hsa (gcn-tables.opt): Generate file.
	(gcn-device-macros.h): Generate file.
	* config/gcn/t-omp-device: Generate isa list from gcn-devices.def.
	* config/gcn/gcn-devices.def: New file.
	* config/gcn/gcn-tables.opt: New file.
	* config/gcn/gcn-tables.opt.urls: New file.
	* config/gcn/gen-gcn-device-macros.awk: New file.
	* config/gcn/gen-opt-tables.awk: New file.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (EF_AMDGPU_MACH): Generate from gcn-devices.def.
	(gcn_gfx803_s): Delete.
	(gcn_gfx900_s): Delete.
	(gcn_gfx906_s): Delete.
	(gcn_gfx908_s): Delete.
	(gcn_gfx90a_s): Delete.
	(gcn_gfx90c_s): Delete.
	(gcn_gfx1030_s): Delete.
	(gcn_gfx1036_s): Delete.
	(gcn_gfx1100_s): Delete.
	(gcn_gfx1103_s): Delete.
	(gcn_isa_name_len): Delete.
	(isa_hsa_name): Rename ...
	(isa_name): ... to this, and rework using gcn-devices.def.
	(isa_gcc_name): Delete.
	(isa_code): Rework using gcn-devices.def.
	(max_isa_vgprs): Rework using gcn-devices.def.
	(isa_matches_agent): Update isa_name usage.
	(GOMP_OFFLOAD_init_device): Improve diagnostic using the name.
 2024-10-22 11:07:05 +00:00
Tobias Burnus b752eed3e3 OpenMP: Add support for 'self_maps' to the 'require' directive 
'self_maps' implies 'unified_shared_memory', except that the latter
also permits that explicit maps copy data to device memory while
self_maps does not. In GCC, currently, both are handled identical.

gcc/c/ChangeLog:

	* c-parser.cc (c_parser_omp_requires): Handle self_maps clause.

gcc/cp/ChangeLog:

	* parser.cc (cp_parser_omp_requires): Handle self_maps clause.

gcc/fortran/ChangeLog:

	* gfortran.h (enum gfc_omp_requires_kind): Add OMP_REQ_SELF_MAPS.
	(gfc_namespace): Enlarge omp_requires bitfield.
	* module.cc (enum ab_attribute, attr_bits): Add AB_OMP_REQ_SELF_MAPS.
	(mio_symbol_attribute): Handle it.
	* openmp.cc (gfc_check_omp_requires, gfc_match_omp_requires): Handle
	self_maps clause.
	* parse.cc (gfc_parse_file): Handle self_maps clause.

gcc/ChangeLog:

	* lto-cgraph.cc (output_offload_tables, omp_requires_to_name): Handle
	self_maps clause.
	* omp-general.cc (struct omp_ts_info, omp_context_selector_matches):
	Likewise for the associated trait.
	* omp-general.h (enum omp_requires): Add OMP_REQUIRES_SELF_MAPS.
	* omp-selectors.h (enum omp_ts_code): Add
	OMP_TRAIT_IMPLEMENTATION_SELF_MAPS.

include/ChangeLog:

	* gomp-constants.h (GOMP_REQUIRES_SELF_MAPS): #define.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_num_devices):
	Accept self_maps clause.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_get_num_devices):
	Likewise.
	* libgomp.texi (TR13 Impl. Status): Set to 'Y'.
	* target.c (gomp_requires_to_name, GOMP_offload_register_ver,
	gomp_target_init): Handle self_maps clause.
	* testsuite/libgomp.fortran/self_maps.f90: New test.

gcc/testsuite/ChangeLog:

	* c-c++-common/gomp/declare-variant-1.c: Add self_maps test.
	* c-c++-common/gomp/requires-4.c: Likewise.
	* gfortran.dg/gomp/declare-variant-3.f90:  Likewise.
	* c-c++-common/gomp/requires-2.c: Update dg-error msg.
	* gfortran.dg/gomp/requires-2.f90: Likewise.
	* gfortran.dg/gomp/requires-self-maps-aux.f90: New.
	* gfortran.dg/gomp/requires-self-maps.f90: New.
 2024-09-24 10:53:59 +02:00
Tobias Burnus cdb9aa0f62 OpenMP: Fix omp_get_device_from_uid, minor cleanup 
In Fortran, omp_get_device_from_uid can also accept substrings, which are
then not NUL terminated.  Fixed by introducing a fortran.c wrapper function.
Additionally, in case of a fail the plugin functions now return NULL instead
of failing fatally such that a fall-back UID is generated.

gcc/ChangeLog:

	* omp-general.cc (omp_runtime_api_procname): Strip "omp_" from
	string; move get_device_from_uid as now a '_' suffix exists.

libgomp/ChangeLog:

	* fortran.c (omp_get_device_from_uid_): New function.
	* libgomp.map (GOMP_6.0): Add it.
	* oacc-host.c (host_dispatch): Init '.uid' and '.get_uid_func'.
	* omp_lib.f90.in: Make it used by removing bind(C).
	* omp_lib.h.in: Likewise.
	* target.c (omp_get_device_from_uid): Ensure the device is initialized.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_uid): Add function comment;
	return NULL in case of an error.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_get_uid): Likewise.
	* testsuite/libgomp.fortran/device_uid.f90: Update to test substrings.
 2024-09-23 15:58:39 +02:00
Tobias Burnus bf4a5efa80 OpenMP: Add get_device_from_uid/omp_get_uid_from_device routines 
Those TR13/OpenMP 6.0 routines permit a reproducible offloading to
a specific device by mapping an OpenMP device number to a
unique ID (UID). The GPU device UIDs should be universally unique,
the one for the host is not.

gcc/ChangeLog:

	* omp-general.cc (omp_runtime_api_procname): Add
	get_device_from_uid and omp_get_uid_from_device routines.

include/ChangeLog:

	* cuda/cuda.h (cuDeviceGetUuid): Declare.
	(cuDeviceGetUuid_v2): Add prototype.

libgomp/ChangeLog:

	* config/gcn/target.c (omp_get_uid_from_device,
	omp_get_device_from_uid): Add stub implementation.
	* config/nvptx/target.c (omp_get_uid_from_device,
	omp_get_device_from_uid): Likewise.
	* fortran.c (omp_get_uid_from_device_,
	omp_get_uid_from_device_8_): New functions.
	* libgomp-plugin.h (GOMP_OFFLOAD_get_uid): Add prototype.
	* libgomp.h (struct gomp_device_descr): Add 'uid' and 'get_uid_func'.
	* libgomp.map (GOMP_6.0): New, includind the new UID routines.
	* libgomp.texi (OpenMP Technical Report 13): Mark UID routines as 'Y'.
	(Device Information Routines): Document new UID routines.
	(Offload-Target Specifics): Document UID format.
	* omp.h.in (omp_get_device_from_uid, omp_get_uid_from_device):
	New prototype.
	* omp_lib.f90.in (omp_get_device_from_uid, omp_get_uid_from_device):
	New interface.
	* omp_lib.h.in: Likewise.
	* plugin/cuda-lib.def: Add cuDeviceGetUuid and cuDeviceGetUuid_v2 via
	CUDA_ONE_CALL_MAYBE_NULL.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_uid): New.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_get_uid): New.
	* target.c (str_omp_initial_device): New static var.
	(STR_OMP_DEV_PREFIX): Define.
	(gomp_get_uid_for_device, omp_get_uid_from_device,
	omp_get_device_from_uid): New.
	(gomp_load_plugin_for_device): DLSYM_OPT the function 'get_uid'.
	(gomp_target_init): Set the device's 'uid' field to NULL.
	* testsuite/libgomp.c/device_uid.c: New test.
	* testsuite/libgomp.fortran/device_uid.f90: New test.
 2024-09-20 09:25:33 +02:00
Tobias Burnus 18f477980c libgomp: Enable USM for AMD APUs and MI200 devices 
If HSA_AMD_SYSTEM_INFO_SVM_ACCESSIBLE_BY_DEFAULT is true,
all GPUs on the system support unified shared memory. That's
the case for APUs and MI200 devices when XNACK is enabled.

XNACK can be enabled by setting HSA_XNACK=1 as env var for
supported devices; otherwise, if disable, USM code will
use host fallback.

gcc/ChangeLog:

	* config/gcn/gcn-hsa.h (gcn_local_sym_hash): Fix typo.

include/ChangeLog:

	* hsa.h (HSA_AMD_SYSTEM_INFO_SVM_ACCESSIBLE_BY_DEFAULT): Add
	enum value.

libgomp/ChangeLog:

	* libgomp.texi (gcn): Update USM handling
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_num_devices): Handle
	USM if HSA_AMD_SYSTEM_INFO_SVM_ACCESSIBLE_BY_DEFAULT is true.
 2024-05-29 15:29:06 +02:00
Frederik Harwath b8e9fd535d amdgcn: Add gfx90c target 
Add support for gfx90c GCN5 APU integrated graphics devices.
The LLVM AMDGPU documentation does not list those devices as supported
by rocm-amdhsa, but it passes most libgomp offloading tests.
Although they are constrainted compared to dGPUs, they might be
interesting for learning, experimentation, and testing.

gcc/ChangeLog:

	* config.gcc: Add gfx90c.
	* config/gcn/gcn-hsa.h (NO_SRAM_ECC): Likewise.
	* config/gcn/gcn-opts.h (enum processor_type): Likewise.
	(TARGET_GFX90c): New macro.
	* config/gcn/gcn.cc (gcn_option_override): Handle gfx90c.
	(gcn_omp_device_kind_arch_isa): Likewise.
	(output_file_start): Likewise.
	* config/gcn/gcn.h: Add gfx90c.
	* config/gcn/gcn.opt: Likewise.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX90c): New macro.
	(get_arch): Handle gfx90c.
	(main): Handle EF_AMDGPU_MACH_AMDGCN_GFX90c
	* config/gcn/t-omp-device: Add gfx90c.
	* doc/install.texi: Likewise.
	* doc/invoke.texi: Likewise.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (isa_hsa_name): Handle EF_AMDGPU_MACH_AMDGCN_GFX90c.
	(isa_code): Handle gfx90c.
	(max_isa_vgprs): Handle EF_AMDGPU_MACH_AMDGCN_GFX90c.

Signed-off-by: Frederik Harwath <frederik@harwath.name>
 2024-04-26 11:23:43 +02:00
Thomas Schwinge a02d7f0edc GCN, nvptx: Errors during device probing are fatal 
Currently, we silently disable libgomp GCN and nvptx plugins/devices in
presence of certain error conditions during device probing, thus typically
silently resorting to host-fallback execution.  Make such errors fatal, similar
as for any other device access later on, so that we early and reliably notice
when things go wrong.  (Keep just two cases non-fatal: (a) libgomp GCN or nvptx
plugins are available but 'libhsa-runtime64.so.1' or 'libcuda.so.1' are not,
and (b) those are available, but the corresponding devices are not.)

This resolves the issue that we've got execution test cases unexpectedly
PASSing, despite:

    libgomp: GCN fatal error: Run-time could not be initialized
    Runtime message: HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.

..., and therefore they were not offloaded to the GCN device, but ran in
host-fallback execution mode.  What happend in that scenario is that in
'init_hsa_context' during the initial 'GOMP_OFFLOAD_get_num_devices' we ran
into 'HSA_STATUS_ERROR_OUT_OF_RESOURCES', but it wasn't fatal, but just
silently disabled the libgomp plugin/device.

Especially "entertaining" were cases where such unintended host-fallback
execution happened during effective-target checks like
'offload_device_available' (host-fallback execution there meaning: no offload
device available), but actual test cases then were running with an offload
device available, and therefore mis-configured.

	include/
	* cuda/cuda.h (CUresult): Add 'CUDA_ERROR_NO_DEVICE'.
	libgomp/
	* plugin/plugin-gcn.c (init_hsa_context): Add and handle
	'bool probe' parameter.  Adjust all users; errors during device
	probing are fatal.
	* plugin/plugin-nvptx.c (nvptx_get_num_devices): Aside from
	'CUDA_ERROR_NO_DEVICE', errors during device probing are fatal.
 2024-04-08 22:08:00 +02:00
Richard Biener 78b56a12dd amdgcn: Add gfx1036 target 
Add support for the gfx1036 RDNA2 APU integrated graphics devices.  The ROCm
documentation warns that these may not be supported, but it seems to work
at least partially.

gcc/ChangeLog:

	* config.gcc (amdgcn): Add gfx1036 entries.
	* config/gcn/gcn-hsa.h (NO_XNACK): Likewise.
	(gcn_local_sym_hash): Likewise.
	* config/gcn/gcn-opts.h (enum processor_type): Likewise.
	(TARGET_GFX1036): New macro.
	* config/gcn/gcn.cc (gcn_option_override): Handle gfx1036.
	(gcn_omp_device_kind_arch_isa): Likewise.
	(output_file_start): Likewise.
	* config/gcn/gcn.h (TARGET_CPU_CPP_BUILTINS): Add __gfx1036__.
	(TARGET_CPU_CPP_BUILTINS): Rename __gfx1030 to __gfx1030__.
	* config/gcn/gcn.opt: Add gfx1036.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX1036): New.
	(main): Handle gfx1036.
	* config/gcn/t-omp-device: Add gfx1036 isa.
	* doc/install.texi (amdgcn): Add gfx1036.
	* doc/invoke.texi (-march): Likewise.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (EF_AMDGPU_MACH): GFX1036.
	(gcn_gfx1103_s): New.
	(isa_hsa_name): Handle gfx1036.
	(isa_code): Likewise.
	(max_isa_vgprs): Likewise.
 2024-03-25 15:54:37 +01:00
Andrew Stubbs 1bf18629c5 amdgcn: Add gfx1103 target 
Add support for the gfx1103 RDNA3 APU integrated graphics devices.  The ROCm
documentation warns that these may not be supported, but it seems to work
at least partially.

gcc/ChangeLog:

	* config.gcc (amdgcn): Add gfx1103 entries.
	* config/gcn/gcn-hsa.h (NO_XNACK): Likewise.
	(gcn_local_sym_hash): Likewise.
	* config/gcn/gcn-opts.h (enum processor_type): Likewise.
	(TARGET_GFX1103): New macro.
	* config/gcn/gcn.cc (gcn_option_override): Handle gfx1103.
	(gcn_omp_device_kind_arch_isa): Likewise.
	(output_file_start): Likewise.
	(gcn_hsa_declare_function_name): Use TARGET_RDNA3, not just gfx1100.
	* config/gcn/gcn.h (TARGET_CPU_CPP_BUILTINS): Add __gfx1103__.
	* config/gcn/gcn.opt: Add gfx1103.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX1103): New.
	(main): Handle gfx1103.
	* config/gcn/t-omp-device: Add gfx1103 isa.
	* doc/install.texi (amdgcn): Add gfx1103.
	* doc/invoke.texi (-march): Likewise.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (EF_AMDGPU_MACH): GFX1103.
	(gcn_gfx1103_s): New.
	(isa_hsa_name): Handle gfx1103.
	(isa_code): Likewise.
	(max_isa_vgprs): Likewise.
 2024-03-22 14:45:15 +00:00
Thomas Schwinge 84fc8f4f32 GCN: The original meaning of 'GCN_SUPPRESS_HOST_FALLBACK' isn't applicable (non-shared memory system) 
'GCN_SUPPRESS_HOST_FALLBACK' originated as 'HSA_SUPPRESS_HOST_FALLBACK' in the
libgomp HSA plugin, where the idea was -- in my understanding -- that you
wouldn't have device code available for all functions that may be called, and
in that case transparently (shared memory system!) do host-fallback execution.
Or, with 'HSA_SUPPRESS_HOST_FALLBACK' set, you'd get those diagnosed.

This has then been copied into the libgomp GCN plugin as
'GCN_SUPPRESS_HOST_FALLBACK'.  However, the original meaning isn't applicable
for the libgomp GCN plugin anymore: we assume that we're generating device code
for all relevant functions, and we're implementing a non-shared memory system,
where we cannot transparently do host-fallback execution for individual
functions.

However, 'GCN_SUPPRESS_HOST_FALLBACK' has gained an additional meaning, to
enforce a fatal error in case that 'libhsa-runtime64.so.1' can't be dynamically
loaded; keep that meaning.

	libgomp/
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_can_run): Don't consider
	'GCN_SUPPRESS_HOST_FALLBACK' anymore (assume always-'true').
	(init_hsa_context): Adjust 'GCN_SUPPRESS_HOST_FALLBACK' error
	message.
 2024-03-08 16:35:28 +01:00
Thomas Schwinge ab70addf56 GCN, nvptx: Fatal error for missing symbols in 'libhsa-runtime64.so.1', 'libcuda.so.1' 
If 'libhsa-runtime64.so.1', 'libcuda.so.1' are not available, the corresponding
libgomp plugin/device gets disabled, as before.  But if they are available,
report any inconsistencies such as missing symbols, similar to how we fail in
presence of other issues during device initialization.

	libgomp/
	* plugin/plugin-gcn.c (init_hsa_runtime_functions): Fatal error
	for missing symbols.
	* plugin/plugin-nvptx.c (init_cuda_lib): Likewise.
 2024-03-08 16:35:28 +01:00
Richard Biener c34ab549d8 Avoid registering unsupported OMP offload devices 
The following avoids registering unsupported GCN offload devices
when iterating over available ones.  With a Zen4 desktop CPU
you will have an IGPU (unspported) which will otherwise be made
available.  This causes testcases like
libgomp.c-c++-common/non-rect-loop-1.c which iterate over all
decives to FAIL.

libgomp/
	* plugin/plugin-gcn.c (suitable_hsa_agent_p): Filter out
	agents with unsupported ISA.
 2024-01-26 15:36:35 +01:00
Richard Biener 209ed06c3a Fix architecture support in OMP_OFFLOAD_init_device for gcn 
The following makes the existing architecture support check work
instead of being optimized away (enum vs. -1).  This avoids
later asserts when we assume such devices are never actually
used.

libgomp/
	* plugin/plugin-gcn.c
	(EF_AMDGPU_MACH::EF_AMDGPU_MACH_UNSUPPORTED): Add.
	(isa_code): Return that instead of -1.
	(GOMP_OFFLOAD_init_device): Adjust.
 2024-01-26 15:36:35 +01:00
Andrew Stubbs 99890e1552 amdgcn: additional gfx1030/gfx1100 support 
This is enough to get gfx1030 and gfx1100 working; there are still some test
failures to investigate, and probably some tuning to do.

gcc/ChangeLog:

	* config/gcn/gcn-opts.h (TARGET_PACKED_WORK_ITEMS): Add TARGET_RDNA3.
	* config/gcn/gcn-valu.md (all_convert): New iterator.
	(<convop><V_INT_1REG_ALT:mode><V_INT_1REG:mode>2<exec>): New
	define_expand, and rename the old one to ...
	(*<convop><V_INT_1REG_ALT:mode><V_INT_1REG:mode>_sdwa<exec>): ... this.
	(extend<V_INT_1REG_ALT:mode><V_INT_1REG:mode>2<exec>): Likewise, to ...
	(extend<V_INT_1REG_ALT:mode><V_INT_1REG:mode>_sdwa<exec>): .. this.
	(*<convop><V_INT_1REG_ALT:mode><V_INT_1REG:mode>_shift<exec>): New.
	* config/gcn/gcn.cc (gcn_global_address_p): Use "offsetbits" correctly.
	(gcn_hsa_declare_function_name): Update the vgpr counting for gfx1100.
	* config/gcn/gcn.md (<u>mulhisi3): Disable on RDNA3.
	(<u>mulqihi3_scalar): Likewise.

libgcc/ChangeLog:

	* config/gcn/amdgcn_veclib.h (CDNA3_PLUS): Handle RDNA3.

libgomp/ChangeLog:

	* config/gcn/time.c (RTC_TICKS): Configure RDNA3.
	(omp_get_wtime): Add RDNA3-compatible variant.
	* plugin/plugin-gcn.c (max_isa_vgprs): Tune for gfx1030 and gfx1100.

Signed-off-by:  Andrew Stubbs <ams@baylibre.com>
 2024-01-26 11:38:47 +00:00
Thomas Schwinge f9290cdf46 GCN: Add pre-initial support for gfx1100: 'EF_AMDGPU_MACH_AMDGCN_GFX1100' 
../../../source-gcc/libgomp/plugin/plugin-gcn.c: In function ‘isa_hsa_name’:
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c:1666:10: error: ‘EF_AMDGPU_MACH_AMDGCN_GFX1100’ undeclared (first use in this function); did you mean ‘EF_AMDGPU_MACH_AMDGCN_GFX1030’?
     1666 |     case EF_AMDGPU_MACH_AMDGCN_GFX1100:
          |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          |          EF_AMDGPU_MACH_AMDGCN_GFX1030
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c:1666:10: note: each undeclared identifier is reported only once for each function it appears in
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c: In function ‘isa_code’:
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c:1711:12: error: ‘EF_AMDGPU_MACH_AMDGCN_GFX1100’ undeclared (first use in this function); did you mean ‘EF_AMDGPU_MACH_AMDGCN_GFX1030’?
     1711 |     return EF_AMDGPU_MACH_AMDGCN_GFX1100;
          |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          |            EF_AMDGPU_MACH_AMDGCN_GFX1030
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c: In function ‘max_isa_vgprs’:
    ../../../source-gcc/libgomp/plugin/plugin-gcn.c:1728:10: error: ‘EF_AMDGPU_MACH_AMDGCN_GFX1100’ undeclared (first use in this function); did you mean ‘EF_AMDGPU_MACH_AMDGCN_GFX1030’?
     1728 |     case EF_AMDGPU_MACH_AMDGCN_GFX1100:
          |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          |          EF_AMDGPU_MACH_AMDGCN_GFX1030
    make[4]: *** [Makefile:813: libgomp_plugin_gcn_la-plugin-gcn.lo] Error 1

Fix-up for commit 52a2c659ae
"GCN: Add pre-initial support for gfx1100".

	libgomp/
	* plugin/plugin-gcn.c (EF_AMDGPU_MACH): Add
	'EF_AMDGPU_MACH_AMDGCN_GFX1100'.
 2024-01-08 20:46:37 +01:00
Julian Brown a17299c17a OpenMP: Support accelerated 2D/3D memory copies for AMD GCN 
This patch adds support for 2D/3D memory copies for omp_target_memcpy_rect
using AMD extensions to the HSA API.  This is just the AMD GCN-specific
part of the following patch:

  https://gcc.gnu.org/pipermail/gcc-patches/2023-September/631001.html

2024-01-04  Julian Brown  <julian@codesourcery.com>

libgomp/
	* plugin/plugin-gcn.c (hsa_runtime_fn_info): Add
	hsa_amd_memory_lock_fn, hsa_amd_memory_unlock_fn,
	hsa_amd_memory_async_copy_rect_fn function pointers.
	(init_hsa_runtime_functions): Add above functions, with
	DLSYM_OPT_FN.
	(GOMP_OFFLOAD_memcpy2d, GOMP_OFFLOAD_memcpy3d): New functions.
 2024-01-08 17:56:17 +00:00
Tobias Burnus 52a2c659ae GCN: Add pre-initial support for gfx1100 
ROCm since 5.7.1 supports gfx1100 (RDNA3) cards. This commit adds support
for it, mostly by assuming gfx1100 behaves identical to gfx1030.  Like gfx1030,
gfx1100 support is neither documented nor the build of the multilib enabled by
default.

But contrary to gfx1030, gfx1100 has a known issue causing some libraries not
to build, including newlib: The sdwa variant of v_mov_b32_sdwa is not supported
by the hardware but GCC current does generates this instruction.
This will be addressed in a later commit.

gcc/ChangeLog:

	* config.gcc (amdgcn-*-amdhsa): Accept --with-arch=gfx1100.
	* config/gcn/gcn-hsa.h (NO_XNACK): Add gfx1100:
	(ASM_SPEC): Handle gfx1100.
	* config/gcn/gcn-opts.h (enum processor_type): Add PROCESSOR_GFX1100.
	(enum gcn_isa): Add ISA_RDNA3.
	(TARGET_GFX1100, TARGET_RDNA2_PLUS, TARGET_RDNA3): Define.
	* config/gcn/gcn-valu.md: Change TARGET_RDNA2 to TARGET_RDNA2_PLUS.
	* config/gcn/gcn.cc (gcn_option_override,
	gcn_omp_device_kind_arch_isa, output_file_start): Handle gfx1100.
	(gcn_global_address_p, gcn_addr_space_legitimate_address_p): Change
	TARGET_RDNA2 to TARGET_RDNA2_PLUS.
	(gcn_hsa_declare_function_name): Don't use '.amdhsa_reserve_flat_scratch'
	with gfx1100.
	* config/gcn/gcn.h (ASSEMBLER_DIALECT): Likewise.
	(TARGET_CPU_CPP_BUILTINS): Define __RDNA3__, __gfx1030__ and
	__gfx1100__.
	* config/gcn/gcn.md: Change TARGET_RDNA2 to TARGET_RDNA2_PLUS.
	* config/gcn/gcn.opt (Enum gpu_type): Add gfx1100.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX1100): Define.
	(isa_has_combined_avgprs, main): Handle gfx1100.
	* config/gcn/t-omp-device (isa): Add gfx1100.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (gcn_gfx1100_s): New const string.
	(gcn_isa_name_len): Fix length.
	(isa_hsa_name, isa_code, max_isa_vgprs): Handle gfx1100.
 2024-01-08 15:12:44 +01:00
Jakub Jelinek a945c346f5 Update copyright years. 2024-01-03 12:19:35 +01:00
Andrew Stubbs e7d6c277fa amdgcn, libgomp: low-latency allocator 
This implements the OpenMP low-latency memory allocator for AMD GCN using the
small per-team LDS memory (Local Data Store).

Since addresses can now refer to LDS space, the "Global" address space is
no-longer compatible.  This patch therefore switches the backend to use
entirely "Flat" addressing (which supports both memories).  A future patch
will re-enable "global" instructions for cases where it is known to be safe
to do so.

gcc/ChangeLog:

	* config/gcn/gcn-builtins.def (DISPATCH_PTR): New built-in.
	* config/gcn/gcn.cc (gcn_init_machine_status): Disable global
	addressing.
	(gcn_expand_builtin_1): Implement GCN_BUILTIN_DISPATCH_PTR.

libgomp/ChangeLog:

	* config/gcn/libgomp-gcn.h (TEAM_ARENA_START): Move to here.
	(TEAM_ARENA_FREE): Likewise.
	(TEAM_ARENA_END): Likewise.
	(GCN_LOWLAT_HEAP): New.
	* config/gcn/team.c (LITTLEENDIAN_CPU): New, and import hsa.h.
	(__gcn_lowlat_init): New prototype.
	(gomp_gcn_enter_kernel): Initialize the low-latency heap.
	* libgomp.h (TEAM_ARENA_START): Move to libgomp.h.
	(TEAM_ARENA_FREE): Likewise.
	(TEAM_ARENA_END): Likewise.
	* plugin/plugin-gcn.c (lowlat_size): New variable.
	(print_kernel_dispatch): Label the group_segment_size purpose.
	(init_environment_variables): Read GOMP_GCN_LOWLAT_POOL.
	(create_kernel_dispatch): Pass low-latency head allocation to kernel.
	(run_kernel): Use shadow; don't assume values.
	* testsuite/libgomp.c/omp_alloc-traits.c: Enable for amdgcn.
	* config/gcn/allocator.c: New file.
	* libgomp.texi: Document low-latency implementation details.
 2023-12-06 16:48:57 +00:00
Andrew StubbsandAndrew Jenner ae0d2c2402 amdgcn: Add Accelerator VGPR registers 
Add the new CDNA register file.  We don't support any of the specialized
instructions that use these registers, but they're useful to relieve
register pressure without spilling to stack.

Co-authored-by: Andrew Jenner  <andrew@codesourcery.com>

gcc/ChangeLog:

	* config/gcn/constraints.md: Add "a" AVGPR constraint.
	* config/gcn/gcn-valu.md (*mov<mode>): Add AVGPR alternatives.
	(*mov<mode>_4reg): Likewise.
	(@mov<mode>_sgprbase): Likewise.
	(gather<mode>_insn_1offset<exec>): Likewise.
	(gather<mode>_insn_1offset_ds<exec>): Likewise.
	(gather<mode>_insn_2offsets<exec>): Likewise.
	(scatter<mode>_expr<exec_scatter>): Likewise.
	(scatter<mode>_insn_1offset_ds<exec_scatter>): Likewise.
	(scatter<mode>_insn_2offsets<exec_scatter>): Likewise.
	* config/gcn/gcn.cc (MAX_NORMAL_AVGPR_COUNT): Define.
	(gcn_class_max_nregs): Handle AVGPR_REGS and ALL_VGPR_REGS.
	(gcn_hard_regno_mode_ok): Likewise.
	(gcn_regno_reg_class): Likewise.
	(gcn_spill_class): Allow spilling to AVGPRs on TARGET_CDNA1_PLUS.
	(gcn_sgpr_move_p): Handle AVGPRs.
	(gcn_secondary_reload): Reload AVGPRs via VGPRs.
	(gcn_conditional_register_usage): Handle AVGPRs.
	(gcn_vgpr_equivalent_register_operand): New function.
	(gcn_valid_move_p): Check for validity of AVGPR moves.
	(gcn_compute_frame_offsets): Handle AVGPRs.
	(gcn_memory_move_cost): Likewise.
	(gcn_register_move_cost): Likewise.
	(gcn_vmem_insn_p): Handle TYPE_VOP3P_MAI.
	(gcn_md_reorg): Handle AVGPRs.
	(gcn_hsa_declare_function_name): Likewise.
	(print_reg): Likewise.
	(gcn_dwarf_register_number): Likewise.
	* config/gcn/gcn.h (FIRST_AVGPR_REG): Define.
	(AVGPR_REGNO): Define.
	(LAST_AVGPR_REG): Define.
	(SOFT_ARG_REG): Update.
	(FRAME_POINTER_REGNUM): Update.
	(DWARF_LINK_REGISTER): Update.
	(FIRST_PSEUDO_REGISTER): Update.
	(AVGPR_REGNO_P): Define.
	(enum reg_class): Add AVGPR_REGS and ALL_VGPR_REGS.
	(REG_CLASS_CONTENTS): Add new register classes and add entries for
	AVGPRs to all classes.
	(REGISTER_NAMES): Add AVGPRs.
	* config/gcn/gcn.md (FIRST_AVGPR_REG, LAST_AVGPR_REG): Define.
	(AP_REGNUM, FP_REGNUM): Update.
	(define_attr "type"): Add vop3p_mai.
	(define_attr "unit"): Handle vop3p_mai.
	(define_attr "gcn_version"): Add "cdna2".
	(define_attr "enabled"): Handle cdna2.
	(*mov<mode>_insn): Add AVGPR alternatives.
	(*movti_insn): Likewise.
	* config/gcn/mkoffload.cc (isa_has_combined_avgprs): New.
	(process_asm): Process avgpr_count.
	* config/gcn/predicates.md (gcn_avgpr_register_operand): New.
	(gcn_avgpr_hard_register_operand): New.
	* doc/md.texi: Document the "a" constraint.

gcc/testsuite/ChangeLog:

	* gcc.target/gcn/avgpr-mem-double.c: New test.
	* gcc.target/gcn/avgpr-mem-int.c: New test.
	* gcc.target/gcn/avgpr-mem-long.c: New test.
	* gcc.target/gcn/avgpr-mem-short.c: New test.
	* gcc.target/gcn/avgpr-spill-double.c: New test.
	* gcc.target/gcn/avgpr-spill-int.c: New test.
	* gcc.target/gcn/avgpr-spill-long.c: New test.
	* gcc.target/gcn/avgpr-spill-short.c: New test.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (max_isa_vgprs): New.
	(run_kernel): CDNA2 devices have more VGPRs.
 2023-11-15 14:02:00 +00:00
Kwok Cheung Yeung a49c7d3193 openmp: Add support for the 'indirect' clause in C/C++ 
This adds support for the 'indirect' clause in the 'declare target'
directive.  Functions declared as indirect may be called via function
pointers passed from the host in offloaded code.

Virtual calls to member functions via the object pointer in C++ are
currently not supported in target regions.

2023-11-07  Kwok Cheung Yeung  <kcy@codesourcery.com>

gcc/c-family/
	* c-attribs.cc (c_common_attribute_table): Add attribute for
	indirect functions.
	* c-pragma.h (enum parma_omp_clause): Add entry for indirect clause.

gcc/c/
	* c-decl.cc (c_decl_attributes): Add attribute for indirect
	functions.
	* c-lang.h (c_omp_declare_target_attr): Add indirect field.
	* c-parser.cc (c_parser_omp_clause_name): Handle indirect clause.
	(c_parser_omp_clause_indirect): New.
	(c_parser_omp_all_clauses): Handle indirect clause.
	(OMP_DECLARE_TARGET_CLAUSE_MASK): Add indirect clause to mask.
	(c_parser_omp_declare_target): Handle indirect clause.  Emit error
	message if device_type or indirect clauses used alone.  Emit error
	if indirect clause used with device_type that is not 'any'.
	(OMP_BEGIN_DECLARE_TARGET_CLAUSE_MASK): Add indirect clause to mask.
	(c_parser_omp_begin): Handle indirect clause.
	* c-typeck.cc (c_finish_omp_clauses): Handle indirect clause.

gcc/cp/
	* cp-tree.h (cp_omp_declare_target_attr): Add indirect field.
	* decl2.cc (cplus_decl_attributes): Add attribute for indirect
	functions.
	* parser.cc (cp_parser_omp_clause_name): Handle indirect clause.
	(cp_parser_omp_clause_indirect): New.
	(cp_parser_omp_all_clauses): Handle indirect clause.
	(handle_omp_declare_target_clause): Add extra parameter.  Add
	indirect attribute for indirect functions.
	(OMP_DECLARE_TARGET_CLAUSE_MASK): Add indirect clause to mask.
	(cp_parser_omp_declare_target): Handle indirect clause.  Emit error
	message if device_type or indirect clauses used alone.  Emit error
	if indirect clause used with device_type that is not 'any'.
	(OMP_BEGIN_DECLARE_TARGET_CLAUSE_MASK): Add indirect clause to mask.
	(cp_parser_omp_begin): Handle indirect clause.
	* semantics.cc (finish_omp_clauses): Handle indirect clause.

gcc/
	* lto-cgraph.cc (enum LTO_symtab_tags): Add tag for indirect
	functions.
	(output_offload_tables): Write indirect functions.
	(input_offload_tables): read indirect functions.
	* lto-section-names.h (OFFLOAD_IND_FUNC_TABLE_SECTION_NAME): New.
	* omp-builtins.def (BUILT_IN_GOMP_TARGET_MAP_INDIRECT_PTR): New.
	* omp-offload.cc (offload_ind_funcs): New.
	(omp_discover_implicit_declare_target): Add functions marked with
	'omp declare target indirect' to indirect functions list.
	(omp_finish_file): Add indirect functions to section for offload
	indirect functions.
	(execute_omp_device_lower): Redirect indirect calls on target by
	passing function pointer to BUILT_IN_GOMP_TARGET_MAP_INDIRECT_PTR.
	(pass_omp_device_lower::gate): Run pass_omp_device_lower if
	indirect functions are present on an accelerator device.
	* omp-offload.h (offload_ind_funcs): New.
	* tree-core.h (omp_clause_code): Add OMP_CLAUSE_INDIRECT.
	* tree.cc (omp_clause_num_ops): Add entry for OMP_CLAUSE_INDIRECT.
	(omp_clause_code_name): Likewise.
	* tree.h (OMP_CLAUSE_INDIRECT_EXPR): New.
	* config/gcn/mkoffload.cc (process_asm): Process offload_ind_funcs
	section.  Count number of indirect functions.
	(process_obj): Emit number of indirect functions.
	* config/nvptx/mkoffload.cc (ind_func_ids, ind_funcs_tail): New.
	(process): Emit offload_ind_func_table in PTX code.  Emit indirect
	function names and count in image.
	* config/nvptx/nvptx.cc (nvptx_record_offload_symbol): Mark
	indirect functions in PTX code with IND_FUNC_MAP.

gcc/testsuite/
	* c-c++-common/gomp/declare-target-7.c: Update expected error message.
	* c-c++-common/gomp/declare-target-indirect-1.c: New.
	* c-c++-common/gomp/declare-target-indirect-2.c: New.
	* g++.dg/gomp/attrs-21.C (v12): Update expected error message.
	* g++.dg/gomp/declare-target-indirect-1.C: New.
	* gcc.dg/gomp/attrs-21.c (v12): Update expected error message.

include/
	* gomp-constants.h (GOMP_VERSION): Increment to 3.
	(GOMP_VERSION_SUPPORTS_INDIRECT_FUNCS): New.

libgcc/
	* offloadstuff.c (OFFLOAD_IND_FUNC_TABLE_SECTION_NAME): New.
	(__offload_ind_func_table): New.
	(__offload_ind_funcs_end): New.
	(__OFFLOAD_TABLE__): Add entries for indirect functions.

libgomp/
	* Makefile.am (libgomp_la_SOURCES): Add target-indirect.c.
	* Makefile.in: Regenerate.
	* libgomp-plugin.h (GOMP_INDIRECT_ADDR_MAP): New define.
	(GOMP_OFFLOAD_load_image): Add extra argument.
	* libgomp.h (struct indirect_splay_tree_key_s): New.
	(indirect_splay_tree_node, indirect_splay_tree,
	indirect_splay_tree_key): New.
	(indirect_splay_compare): New.
	* libgomp.map (GOMP_5.1.1): Add GOMP_target_map_indirect_ptr.
	* libgomp.texi (OpenMP 5.1): Update documentation on indirect
	calls in target region and on indirect clause.
	(Other new OpenMP 5.2 features): Add entry for virtual function calls.
	* libgomp_g.h (GOMP_target_map_indirect_ptr): Add prototype.
	* oacc-host.c (host_load_image): Add extra argument.
	* target.c (gomp_load_image_to_device): If the GOMP_VERSION is high
	enough, read host indirect functions table and pass to
	load_image_func.
	* config/accel/target-indirect.c: New.
	* config/linux/target-indirect.c: New.
	* config/gcn/team.c (build_indirect_map): Add prototype.
	(gomp_gcn_enter_kernel): Initialize support for indirect
	function calls on GCN target.
	* config/nvptx/team.c (build_indirect_map): Add prototype.
	(gomp_nvptx_main): Initialize support for indirect function
	calls on NVPTX target.
	* plugin/plugin-gcn.c (struct gcn_image_desc): Add field for
	indirect functions count.
	(GOMP_OFFLOAD_load_image): Add extra argument.  If the GOMP_VERSION
	is high enough, build address translation table and copy it to target
	memory.
	* plugin/plugin-nvptx.c (nvptx_tdata): Add field for indirect
	functions count.
	(GOMP_OFFLOAD_load_image): Add extra argument.  If the GOMP_VERSION
	is high enough, Build address translation table and copy it to target
	memory.
	* testsuite/libgomp.c-c++-common/declare-target-indirect-1.c: New.
	* testsuite/libgomp.c-c++-common/declare-target-indirect-2.c: New.
	* testsuite/libgomp.c++/declare-target-indirect-1.C: New.
 2023-11-07 15:44:50 +00:00
Andrew Stubbs c7ec7bd1c6 amdgcn: add -march=gfx1030 EXPERIMENTAL 
Accept the architecture configure option and resolve build failures.  This is
enough to build binaries, but I've not got a device to test it on, so there
are probably runtime issues to fix.  The cache control instructions might be
unsafe (or too conservative), and the kernel metadata might be off.  Vector
reductions will need to be reworked for RDNA2.  In principle, it would be
better to use wavefrontsize32 for this architecture, but that would mean
switching everything to allow SImode masks, so wavefrontsize64 it is.

The multilib is not included in the default configuration so either configure
--with-arch=gfx1030 or include it in --with-multilib-list=gfx1030,....

The majority of this patch has no effect on other devices, but changing from
using scalar writes for the exit value to vector writes means we don't need
the scalar cache write-back instruction anywhere (which doesn't exist in RDNA2).

gcc/ChangeLog:

	* config.gcc: Allow --with-arch=gfx1030.
	* config/gcn/gcn-hsa.h (NO_XNACK): gfx1030 does not support xnack.
	(ASM_SPEC): gfx1030 needs -mattr=+wavefrontsize64 set.
	* config/gcn/gcn-opts.h (enum processor_type): Add PROCESSOR_GFX1030.
	(TARGET_GFX1030): New.
	(TARGET_RDNA2): New.
	* config/gcn/gcn-valu.md (@dpp_move<mode>): Disable for RDNA2.
	(addc<mode>3<exec_vcc>): Add RDNA2 syntax variant.
	(subc<mode>3<exec_vcc>): Likewise.
	(<convop><mode><vndi>2_exec): Add RDNA2 alternatives.
	(vec_cmp<mode>di): Likewise.
	(vec_cmp<u><mode>di): Likewise.
	(vec_cmp<mode>di_exec): Likewise.
	(vec_cmp<u><mode>di_exec): Likewise.
	(vec_cmp<mode>di_dup): Likewise.
	(vec_cmp<mode>di_dup_exec): Likewise.
	(reduc_<reduc_op>_scal_<mode>): Disable for RDNA2.
	(*<reduc_op>_dpp_shr_<mode>): Likewise.
	(*plus_carry_dpp_shr_<mode>): Likewise.
	(*plus_carry_in_dpp_shr_<mode>): Likewise.
	* config/gcn/gcn.cc (gcn_option_override): Recognise gfx1030.
	(gcn_global_address_p): RDNA2 only allows smaller offsets.
	(gcn_addr_space_legitimate_address_p): Likewise.
	(gcn_omp_device_kind_arch_isa): Recognise gfx1030.
	(gcn_expand_epilogue): Use VGPRs instead of SGPRs.
	(output_file_start): Configure gfx1030.
	* config/gcn/gcn.h (TARGET_CPU_CPP_BUILTINS): Add __RDNA2__;
	(ASSEMBLER_DIALECT): New.
	* config/gcn/gcn.md (rdna): New define_attr.
	(enabled): Use "rdna" attribute.
	(gcn_return): Remove s_dcache_wb.
	(addcsi3_scalar): Add RDNA2 syntax variant.
	(addcsi3_scalar_zero): Likewise.
	(addptrdi3): Likewise.
	(mulsi3): v_mul_lo_i32 should be v_mul_lo_u32 on all ISA.
	(*memory_barrier): Add RDNA2 syntax variant.
	(atomic_load<mode>): Add RDNA2 cache control variants, and disable
	scalar atomics for RDNA2.
	(atomic_store<mode>): Likewise.
	(atomic_exchange<mode>): Likewise.
	* config/gcn/gcn.opt (gpu_type): Add gfx1030.
	* config/gcn/mkoffload.cc (EF_AMDGPU_MACH_AMDGCN_GFX1030): New.
	(main): Recognise -march=gfx1030.
	* config/gcn/t-omp-device: Add gfx1030 isa.

libgcc/ChangeLog:

	* config/gcn/amdgcn_veclib.h (CDNA3_PLUS): Set false for __RDNA2__.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (EF_AMDGPU_MACH_AMDGCN_GFX1030): New.
	(isa_hsa_name): Recognise gfx1030.
	(isa_code): Likewise.
	* team.c (defined): Remove s_endpgm.
 2023-10-20 12:40:25 +01:00
Tobias Burnus f1af7d65ff libgomp: plugin-gcn - support 'unified_address' 
Effectively, for GCN (as for nvptx) there is a common address space between
host and device, whether being accessible or not. Thus, this commit
permits to use 'omp requires unified_address' with GCN devices.
(nvptx accepts this requirement since r13-3460-g131d18e928a3ea.)

libgomp/

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_num_devices): Regard
	unified_address requirement as supported.
	* libgomp.texi (OpenMP 5.0, AMD Radeon, nvptx): Remove
	'unified_address' from the not-supported requirements.
 2023-06-06 18:06:14 +02:00
Thomas Schwinge 130c2f3c3a libgomp: Simplify OpenMP reverse offload host <-> device memory copy implementation 
... by using the existing 'goacc_asyncqueue' instead of re-coding parts of it.

Follow-up to commit 131d18e928
"libgomp/nvptx: Prepare for reverse-offload callback handling",
and commit ea4b23d9c8
"libgomp: Handle OpenMP's reverse offloads".

	libgomp/
	* target.c (gomp_target_rev): Instead of 'dev_to_host_cpy',
	'host_to_dev_cpy', 'token', take a single 'goacc_asyncqueue'.
	* libgomp.h (gomp_target_rev): Adjust.
	* libgomp-plugin.c (GOMP_PLUGIN_target_rev): Adjust.
	* libgomp-plugin.h (GOMP_PLUGIN_target_rev): Adjust.
	* plugin/plugin-gcn.c (process_reverse_offload): Adjust.
	* plugin/plugin-nvptx.c (rev_off_dev_to_host_cpy)
	(rev_off_host_to_dev_cpy): Remove.
	(GOMP_OFFLOAD_run): Adjust.
 2023-05-08 15:58:05 +02:00
Thomas Schwinge f8332e52a4 Use 'GOMP_MAP_VARS_TARGET' for OpenACC compute constructs [PR90596] 
Thereby considerably simplify the device plugins' 'GOMP_OFFLOAD_openacc_exec',
'GOMP_OFFLOAD_openacc_async_exec' functions: in terms of lines of code, but in
particular conceptually: no more device memory allocation, host to device data
copying, device memory deallocation -- 'GOMP_MAP_VARS_TARGET' does all that for
us.

This depends on commit 2b2340e236
"Allow libgomp 'cbuf' buffering with OpenACC 'async' for 'ephemeral' data",
where I said that "a use will emerge later", which is this one here.

	PR libgomp/90596
	libgomp/
	* target.c (gomp_map_vars_internal): Allow for
	'param_kind == GOMP_MAP_VARS_OPENACC | GOMP_MAP_VARS_TARGET'.
	* oacc-parallel.c (GOACC_parallel_keyed): Pass
	'GOMP_MAP_VARS_TARGET' to 'goacc_map_vars'.
	* plugin/plugin-gcn.c (alloc_by_agent, gcn_exec)
	(GOMP_OFFLOAD_openacc_exec, GOMP_OFFLOAD_openacc_async_exec):
	Adjust, simplify.
	(gomp_offload_free): Remove.
	* plugin/plugin-nvptx.c (nvptx_exec, GOMP_OFFLOAD_openacc_exec)
	(GOMP_OFFLOAD_openacc_async_exec): Adjust, simplify.
	(cuda_free_argmem): Remove.
	* testsuite/libgomp.oacc-c-c++-common/acc_prof-parallel-1.c:
	Adjust.
 2023-03-10 18:05:27 +01:00
Thomas Schwinge 199867d07b Simplify OpenACC 'no_create' clause implementation 
For 'OFFSET_INLINED', 'gomp_map_val' does the right thing, and we may then
simplify the device plugins accordingly.

This is a follow-up to
Subversion r279551 (Git commit a6163563f2)
"Add OpenACC 2.6's no_create",
Subversion r279622 (Git commit 5bcd470bf0)
"Use gomp_map_val for OpenACC host-to-device address translation".

	libgomp/
	* target.c (gomp_map_vars_internal): Use 'OFFSET_INLINED' for
	'GOMP_MAP_IF_PRESENT'.
	* plugin/plugin-gcn.c (gcn_exec, GOMP_OFFLOAD_openacc_exec)
	(GOMP_OFFLOAD_openacc_async_exec): Adjust.
	* plugin/plugin-nvptx.c (nvptx_exec, GOMP_OFFLOAD_openacc_exec)
	(GOMP_OFFLOAD_openacc_async_exec): Likewise.
	* testsuite/libgomp.oacc-c-c++-common/no_create-1.c: Add 'async'
	testing.
	* testsuite/libgomp.oacc-c-c++-common/no_create-2.c: Likewise.
 2023-03-10 15:48:43 +01:00
Thomas Schwinge 649f1939ba Fix OpenACC/GCN 'acc_ev_enqueue_launch_end' position 
For an OpenACC compute construct, we've currently got:

  - [...]
  - acc_ev_enqueue_launch_start
  - launch kernel
  - free memory
  - acc_ev_free
  - acc_ev_enqueue_launch_end

This confused another thing that I'm working on, so I adjusted that to:

  - [...]
  - acc_ev_enqueue_launch_start
  - launch kernel
  - acc_ev_enqueue_launch_end
  - free memory
  - acc_ev_free

Correspondingly, verify 'acc_ev_alloc', 'acc_ev_free' in
'libgomp.oacc-c-c++-common/acc_prof-parallel-1.c'.

	libgomp/
	* plugin/plugin-gcn.c (gcn_exec): Fix 'acc_ev_enqueue_launch_end'
	position.
	* testsuite/libgomp.oacc-c-c++-common/acc_prof-parallel-1.c:
	Verify 'acc_ev_alloc', 'acc_ev_free'.
 2023-03-10 15:05:01 +01:00
Tobias Burnus f84fdb134d libgomp: enable reverse offload for AMDGCN 
libgomp/ChangeLog:

	* libgomp.texi (5.0 Impl. Status, gcn specifics): Update for
	reverse offload.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_get_num_devices): Accept
	reverse-offload requirement.
 2023-02-03 08:33:17 +01:00
Andrew Stubbs f6fff8a6fc amdgcn, libgomp: Manually allocated stacks 
Switch from using stacks in the "private segment" to using a memory block
allocated on the host side.  The primary reason is to permit the reverse
offload implementation to access values located on the device stack, but
there may also be performance benefits, especially with repeated kernel
invocations.

This implementation unifies the stacks with the "team arena" optimization
feature, and now allows both to have run-time configurable sizes.

A new ABI is needed, so all libraries must be rebuilt, and newlib must be
version 4.3.0.20230120 or newer.

gcc/ChangeLog:

	* config/gcn/gcn-run.cc: Include libgomp-gcn.h.
	(struct kernargs): Replace the common content with kernargs_abi.
	(struct heap): Delete.
	(main): Read GCN_STACK_SIZE envvar.
	Allocate space for the device stacks.
	Write the new kernargs fields.
	* config/gcn/gcn.cc (gcn_option_override): Remove stack_size_opt.
	(default_requested_args): Remove PRIVATE_SEGMENT_BUFFER_ARG and
	PRIVATE_SEGMENT_WAVE_OFFSET_ARG.
	(gcn_addr_space_convert): Mask the QUEUE_PTR_ARG content.
	(gcn_expand_prologue): Move the TARGET_PACKED_WORK_ITEMS to the top.
	Set up the stacks from the values in the kernargs, not private.
	(gcn_expand_builtin_1): Match the stack configuration in the prologue.
	(gcn_hsa_declare_function_name): Turn off the private segment.
	(gcn_conditional_register_usage): Ensure QUEUE_PTR is fixed.
	* config/gcn/gcn.h (FIXED_REGISTERS): Fix the QUEUE_PTR register.
	* config/gcn/gcn.opt (mstack-size): Change the description.

include/ChangeLog:

	* gomp-constants.h (GOMP_VERSION_GCN): Bump.

libgomp/ChangeLog:

	* config/gcn/libgomp-gcn.h (DEFAULT_GCN_STACK_SIZE): New define.
	(DEFAULT_TEAM_ARENA_SIZE): New define.
	(struct heap): Move to this file.
	(struct kernargs_abi): Likewise.
	* config/gcn/team.c (gomp_gcn_enter_kernel): Use team arena size from
	the kernargs.
	* libgomp.h: Include libgomp-gcn.h.
	(TEAM_ARENA_SIZE): Remove.
	(team_malloc): Update the error message.
	* plugin/plugin-gcn.c (struct kernargs): Move common content to
	struct kernargs_abi.
	(struct agent_info): Rename team arenas to ephemeral memories.
	(struct team_arena_list): Rename ....
	(struct ephemeral_memories_list): to this.
	(struct heap): Delete.
	(team_arena_size): New variable.
	(stack_size): New variable.
	(print_kernel_dispatch): Update debug messages.
	(init_environment_variables): Read GCN_TEAM_ARENA_SIZE.
	Read GCN_STACK_SIZE.
	(get_team_arena): Rename ...
	(configure_ephemeral_memories): ... to this, and set up stacks.
	(release_team_arena): Rename ...
	(release_ephemeral_memories): ... to this.
	(destroy_team_arenas): Rename ...
	(destroy_ephemeral_memories): ... to this.
	(create_kernel_dispatch): Add num_threads parameter.
	Adjust for kernargs_abi refactor and ephemeral memories.
	(release_kernel_dispatch): Adjust for ephemeral memories.
	(run_kernel): Pass thread-count to create_kernel_dispatch.
	(GOMP_OFFLOAD_init_device): Adjust for ephemeral memories.
	(GOMP_OFFLOAD_fini_device): Adjust for ephemeral memories.

gcc/testsuite/ChangeLog:

	* gcc.c-torture/execute/pr47237.c: Xfail on amdgcn.
	* gcc.dg/builtin-apply3.c: Xfail for amdgcn.
	* gcc.dg/builtin-apply4.c: Xfail for amdgcn.
	* gcc.dg/torture/stackalign/builtin-apply-3.c: Xfail for amdgcn.
	* gcc.dg/torture/stackalign/builtin-apply-4.c: Xfail for amdgcn.
 2023-02-02 11:47:03 +00:00
Jakub Jelinek 83ffe9cde7 Update copyright years. 2023-01-16 11:52:17 +01:00
Marcel Vollweiler 81476bc4f4 OpenMP: omp_get_max_teams, omp_set_num_teams, and omp_{gs}et_teams_thread_limit on offload devices 
This patch adds support for omp_get_max_teams, omp_set_num_teams, and
omp_{gs}et_teams_thread_limit on offload devices. That includes the usage of
device-specific ICV values (specified as environment variables or changed on a
device). In order to reuse device-specific ICV values, a copy back mechanism is
implemented that copies ICV values back from device to the host.

Additionally, a limitation of the number of teams on gcn offload devices is
implemented.  The number of teams is limited by twice the number of compute
units (one team is executed on one compute unit).  This avoids queueing
unnessecary many teams and a corresponding allocation of large amounts of
memory.  Without that limitation the memory allocation for a large number of
user-specified teams can result in an "memory access fault".
A limitation of the number of teams is already also implemented for nvptx
devices (see nvptx_adjust_launch_bounds in libgomp/plugin/plugin-nvptx.c).

gcc/ChangeLog:

	* gimplify.cc (optimize_target_teams): Set initial num_teams_upper
	to "-2" instead of "1" for non-existing num_teams clause in order to
	disambiguate from the case of an existing num_teams clause with value 1.

libgomp/ChangeLog:

	* config/gcn/icv-device.c (omp_get_teams_thread_limit): Added to
	allow processing of device-specific values.
	(omp_set_teams_thread_limit): Likewise.
	(ialias): Likewise.
	* config/nvptx/icv-device.c (omp_get_teams_thread_limit): Likewise.
	(omp_set_teams_thread_limit): Likewise.
	(ialias): Likewise.
	* icv-device.c (omp_get_teams_thread_limit): Likewise.
	(ialias): Likewise.
	(omp_set_teams_thread_limit): Likewise.
	* icv.c (omp_set_teams_thread_limit): Removed.
	(omp_get_teams_thread_limit): Likewise.
	(ialias): Likewise.
	* libgomp.texi: Updated documentation for nvptx and gcn corresponding
	to the limitation of the number of teams.
	* plugin/plugin-gcn.c (limit_teams): New helper function that limits
	the number of teams by twice the number of compute units.
	(parse_target_attributes): Limit the number of teams on gcn offload
	devices.
	* target.c (get_gomp_offload_icvs): Added teams_thread_limit_var
	handling.
	(gomp_load_image_to_device): Added a size check for the ICVs struct
	variable.
	(gomp_copy_back_icvs): New function that is used in GOMP_target_ext to
	copy back the ICV values from device to host.
	(GOMP_target_ext): Update the number of teams and threads in the kernel
	args also considering device-specific values.
	* testsuite/libgomp.c-c++-common/icv-4.c: Fixed an error in the reading
	of OMP_TEAMS_THREAD_LIMIT from the environment.
	* testsuite/libgomp.c-c++-common/icv-5.c: Extended.
	* testsuite/libgomp.c-c++-common/icv-6.c: Extended.
	* testsuite/libgomp.c-c++-common/icv-7.c: Extended.
	* testsuite/libgomp.c-c++-common/icv-9.c: New test.
	* testsuite/libgomp.fortran/icv-5.f90: New test.
	* testsuite/libgomp.fortran/icv-6.f90: New test.

gcc/testsuite/ChangeLog:

	* c-c++-common/gomp/target-teams-1.c: Adapt expected values for
	num_teams from "1" to "-2" in cases without num_teams clause.
	* g++.dg/gomp/target-teams-1.C: Likewise.
	* gfortran.dg/gomp/defaultmap-4.f90: Likewise.
	* gfortran.dg/gomp/defaultmap-5.f90: Likewise.
	* gfortran.dg/gomp/defaultmap-6.f90: Likewise.
 2022-12-06 06:03:50 -08:00
Tobias Burnus 6edcb5dc42 libgomp/gcn: fix/improve struct output 
output.printf_data.(value union) contains text[128], which has the size
of 128 bytes, sufficient for 16 uint64_t variables; hence value_u64[2]
could be extended to value_u64[6] - sufficient for all required arguments
to gomp_target_rev.  Additionally, next_output.printf_data.(msg union)
contained msg_u64 which then is no longer needed and also caused 32bit
vs 64bit alignment issues.

libgomp/
	* config/gcn/libgomp-gcn.h (struct output):
	Remove 'msg_u64' from the union, change
	value_u64[2] to value_u64[6].
	* config/gcn/target.c (GOMP_target_ext): Update accordingly.
	* plugin/plugin-gcn.c (process_reverse_offload, console_output):
	Likewise.
 2022-11-21 15:22:35 +01:00
Tobias Burnus 8c05d8cd43 libgomp/gcn: Prepare for reverse-offload callback handling 
libgomp/ChangeLog:

	* config/gcn/libgomp-gcn.h: New file; contains
	struct output, declared previously in plugin-gcn.c.
	* config/gcn/target.c: Include it.
	(GOMP_ADDITIONAL_ICVS): Declare as extern var.
	(GOMP_target_ext): Handle reverse offload.
	* plugin/plugin-gcn.c: Include libgomp-gcn.h.
	(struct kernargs): Replace struct def by the one
	from libgomp-gcn.h for output_data.
	(process_reverse_offload): New.
	(console_output): Call it.
 2022-11-19 10:36:27 +01:00
Tobias Burnus dfd75bf7e9 GCN: libgomp+mkoffload.cc: Prepare for reverse offload fn lookup 
Add support to GCN for reverse lookup of function name to prepare for
'omp target device(ancestor:1)'.

gcc/ChangeLog:

	* config/gcn/mkoffload.cc (process_asm): Create .offload_func_table,
	similar to pre-existing .offload_var_table.

libgomp/ChangeLog:

	* plugin/plugin-gcn.c (GOMP_OFFLOAD_load_image): Read
	.offload_func_table to populate rev_fn_table when requested.
 2022-09-09 17:43:12 +02:00
Tobias Burnus 0fcc0cf9dc libgomp: Prepare for reverse offload fn lookup 
Prepare for reverse-offloading function-pointer lookup by passing
a rev_fn_table argument to GOMP_OFFLOAD_load_image.

The argument will be NULL, unless GOMP_REQUIRES_REVERSE_OFFLOAD is
requested and devices not supported it, are filtered out.
(Up to and including this commit, no non-host device claims such
support and the caller currently always passes NULL.)

libgomp/ChangeLog:

	* libgomp-plugin.h (GOMP_OFFLOAD_load_image): Add
	'uint64_t **rev_fn_table' argument.
	* oacc-host.c (host_load_image): Likewise.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_load_image): Likewise;
	currently unused.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_load_image): Likewise.
	* target.c (gomp_load_image_to_device): Update call but pass
	NULL for now.

liboffloadmic/ChangeLog:

	* plugin/libgomp-plugin-intelmic.cpp (GOMP_OFFLOAD_load_image):
	Add (unused) uint64_t **rev_fn_table argument.
 2022-09-09 17:37:09 +02:00
Marcel Vollweiler 9f2fca5659 OpenMP, libgomp: Environment variable syntax extension 
This patch considers the environment variable syntax extension for
device-specific variants of environment variables from OpenMP 5.1 (see
OpenMP 5.1 specification, p. 75 and p. 639).  An environment variable (e.g.
OMP_NUM_TEAMS) can have different suffixes:

_DEV (e.g. OMP_NUM_TEAMS_DEV): affects all devices but not the host.
_DEV_<device> (e.g. OMP_NUM_TEAMS_DEV_42): affects only device with
number <device>.
no suffix (e.g. OMP_NUM_TEAMS): affects only the host.

In future OpenMP versions also suffix _ALL will be introduced (see discussion
https://github.com/OpenMP/spec/issues/3179). This is also considered in this
patch:

_ALL (e.g. OMP_NUM_TEAMS_ALL): affects all devices and the host.

The precedence is as follows (descending). For the host:

	1. no suffix
	2. _ALL

For devices:

	1. _DEV_<device>
	2. _DEV
	3. _ALL

That means, _DEV_<device> is used whenever available. Otherwise _DEV is used if
available, and at last _ALL.  If there is no value for any of the variable
variants, default values are used as already implemented before.

This patch concerns parsing (a), storing (b), output (c) and transmission to the
device (d):

(a) The actual number of devices and the numbering are not known when parsing
the environment variables.  Thus all environment variables are iterated and
searched for device-specific ones.
(b) Only configured device-specific variables are stored.  Thus, a linked list
is used.
(c) The output is done in omp_display_env (see specification p. 468f).  Global
ICVs are tagged with [all], see https://github.com/OpenMP/spec/issues/3179.
ICVs which are not global but aren't handled device-specific yet are tagged
with [host].  omp_display_env outputs the initial values of the ICVs.  That is
why a dedicated data structure is introduced for the inital values only
(gomp_initial_icv_list).
(d) Device-specific ICVs are transmitted to the device via GOMP_ADDITIONAL_ICVS.

libgomp/ChangeLog:

	* config/gcn/icv-device.c (omp_get_default_device): Return device-
	specific ICV.
	(omp_get_max_teams): Added for GCN devices.
	(omp_set_num_teams): Likewise.
	(ialias): Likewise.
	* config/nvptx/icv-device.c (omp_get_default_device): Return device-
	specific ICV.
	(omp_get_max_teams): Added for NVPTX devices.
	(omp_set_num_teams): Likewise.
	(ialias): Likewise.
	* env.c (struct gomp_icv_list): New struct to store entries of initial
	ICV values.
	(struct gomp_offload_icv_list): New struct to store entries of device-
	specific ICV values that are copied to the device and back.
	(struct gomp_default_icv_values): New struct to store default values of
	ICVs according to the OpenMP standard.
	(parse_schedule): Generalized for different variants of OMP_SCHEDULE.
	(print_env_var_error): Function that prints an error for invalid values
	for ICVs.
	(parse_unsigned_long_1): Removed getenv.  Generalized.
	(parse_unsigned_long): Likewise.
	(parse_int_1): Likewise.
	(parse_int): Likewise.
	(parse_int_secure): Likewise.
	(parse_unsigned_long_list): Likewise.
	(parse_target_offload): Likewise.
	(parse_bind_var): Likewise.
	(parse_stacksize): Likewise.
	(parse_boolean): Likewise.
	(parse_wait_policy): Likewise.
	(parse_allocator): Likewise.
	(omp_display_env): Extended to output different variants of environment
	variables.
	(print_schedule): New helper function for omp_display_env which prints
	the values of run_sched_var.
	(print_proc_bind): New helper function for omp_display_env which prints
	the values of proc_bind_var.
	(enum gomp_parse_type): Collection of types used for parsing environment
	variables.
	(ENTRY): Preprocess string lengths of environment variables.
	(OMP_VAR_CNT): Preprocess table size.
	(OMP_HOST_VAR_CNT): Likewise.
	(INT_MAX_STR_LEN): Constant for the maximal number of digits of a device
	number.
	(gomp_get_icv_flag): Returns if a flag for a particular ICV is set.
	(gomp_set_icv_flag): Sets a flag for a particular ICV.
	(print_device_specific_icvs): New helper function for omp_display_env to
	print device specific ICV values.
	(get_device_num): New helper function for parse_device_specific.
	Extracts the device number from an environment variable name.
	(get_icv_member_addr): Gets the memory address for a particular member
	of an ICV struct.
	(gomp_get_initial_icv_item): Get a list item of gomp_initial_icv_list.
	(initialize_icvs): New function to initialize a gomp_initial_icvs
	struct.
	(add_initial_icv_to_list): Adds an ICV struct to gomp_initial_icv_list.
	(startswith): Checks if a string starts with a given prefix.
	(initialize_env): Extended to parse the new syntax of environment
	variables.
	* icv-device.c (omp_get_max_teams): Added.
	(ialias): Likewise.
	(omp_set_num_teams): Likewise.
	* icv.c (omp_set_num_teams): Moved to icv-device.c.
	(omp_get_max_teams): Likewise.
	(ialias): Likewise.
	* libgomp-plugin.h (GOMP_DEVICE_NUM_VAR): Removed.
	(GOMP_ADDITIONAL_ICVS): New target-side struct that
	holds the designated ICVs of the target device.
	* libgomp.h (enum gomp_icvs): Collection of ICVs.
	(enum gomp_device_num): Definition of device numbers for _ALL, _DEV, and
	no suffix.
	(enum gomp_env_suffix): Collection of possible suffixes of environment
	variables.
	(struct gomp_initial_icvs): Contains all ICVs for which we need to store
	initial values.
	(struct gomp_default_icv):New struct to hold ICVs for which we need
	to store initial values.
	(struct gomp_icv_list): Definition of a linked list that is used for
	storing ICVs for the devices and also for _DEV, _ALL, and without
	suffix.
	(struct gomp_offload_icvs): New struct to hold ICVs that are copied to
	a device.
	(struct gomp_offload_icv_list): Definition of a linked list that holds
	device-specific ICVs that are copied to devices.
	(gomp_get_initial_icv_item): Get a list item of gomp_initial_icv_list.
	(gomp_get_icv_flag): Returns if a flag for a particular ICV is set.
	* libgomp.texi: Updated.
	* plugin/plugin-gcn.c (GOMP_OFFLOAD_load_image): Extended to read
	further ICVs from the offload image.
	* plugin/plugin-nvptx.c (GOMP_OFFLOAD_load_image): Likewise.
	* target.c (gomp_get_offload_icv_item): Get a list item of
	gomp_offload_icv_list.
	(get_gomp_offload_icvs): New. Returns the ICV values
	depending on the device num and the variable hierarchy.
	(gomp_load_image_to_device): Extended to copy further ICVs to a device.
	* testsuite/libgomp.c-c++-common/icv-5.c: New test.
	* testsuite/libgomp.c-c++-common/icv-6.c: New test.
	* testsuite/libgomp.c-c++-common/icv-7.c: New test.
	* testsuite/libgomp.c-c++-common/icv-8.c: New test.
	* testsuite/libgomp.c-c++-common/omp-display-env-1.c: New test.
	* testsuite/libgomp.c-c++-common/omp-display-env-2.c: New test.
 2022-09-08 10:19:37 -07:00