The graphics part of the APU is connected with the core complex via infinity fabric. They will have to redesign so that the latency from infinity fabric to L3 cache does not become a bottleneck. They have that modularity to since renoir so that they can easily change blocks of the APU as and when required like they did with cezzane (Just changed the CCX block of the APU)
What you are thinking is only what marketing slides have led you to believe.
The Graphics part of the APU is connected to the Northbridge block which is then connected to the Infinity Fabric/DF via IOH (not directly to IF)
Similarly none of the USB/NVME/SATA etc. are connected to the Infinity fabric directly as shown in slides, they are connected to the Northbridge block connected to IOH, which in the case of Desktop is on the IOD and in the case of Mobile is on die.
Infinity Fabric is connected to L3 in AMD CCX design, the L3 is a cache coherent master and is responsible for snooping and maintaining coherency across CCXs.
So Graphics cores cannot access the L3 of the CPUs.
SLC is like the Infinity Cache in current RDNA2 GPUS( whatever is ejected/victimized from L2 goes to Infinity cache which is connected to UMC)
A much better question would be how could AMD design the Infinity cache to act like SLC. Will be interesting to see how AMD address this issue in the future, because currently the GPU goes to NB/IOH then to UMC, and the UMC handles requests from both CCXs and DF fabric masters.
Coherency between CPU cores and GPU CUs is possible using IF however with Trento + Aldebaran but they follow a different topology and address different use case.
This is how it looks like
On Linux this is the enumeration for the LilacKD-RMB SoC
-[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD] Device 14b5
+-00.2 Advanced Micro Devices, Inc. [AMD] Device 14b6
+-04.0 Advanced Micro Devices, Inc. [AMD] Device 14b7
+-04.1-[33-61]--
+-08.0 Advanced Micro Devices, Inc. [AMD] Device 14b7
+-08.1-[62]--+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] Device 1681
--> GPU
| +-00.1 Advanced Micro Devices, Inc. [AMD/ATI] Device 1640
| +-00.2 Advanced Micro Devices, Inc. [AMD] Device 1649
| +-00.3 Advanced Micro Devices, Inc. [AMD] Device 161d
| +-00.4 Advanced Micro Devices, Inc. [AMD] Device 161e
| +-00.5 Advanced Micro Devices, Inc. [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor
| +-00.6 Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller
| \-00.7 Advanced Micro Devices, Inc. [AMD] Raven/Raven2/Renoir Sensor Fusion Hub