dozob

https://ilgpu.net/

https://ilgpu.net/docs/

How a GPU works

00 Setting up ILGPU

ILGPU should work on any 64-bit platform that .NET supports.

I have even used it on the inexpensive nvidia jetson nano with pretty decent cuda performance.

1. Install the most recent .NET SDK for your chosen platform.

2. Create a new C# project: dotnet new console

3. Add the ILGPU package: dotnet add package ILGPU

01 A GPU is not a CPU

CPU | SIMD: Single Instruction Multiple Data

CPUs have a trick for parallel programs called SIMD.

These are a set of instructions that allow you to have one instruction do operations on multiple pieces of data at once.

GPU |  SIMT: Single Instruction Multiple Threads

SIMT is the same idea as SIMD
but instead of just doing the math instructions in parallel
why not do all the instructions in parallel.

using System;
using System.Threading.Tasks;

static void TestParallelFor()
{
    // Load the data
    int[] data = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
    int[] output = new int[10_000];
    
    // Load the action and execute
    Parallel.For(0, output.Length,
        (int i) =>
        {
            output[i] = data[i % data.Length];
        });
}

-

using ILGPU
using ILGPU.Runtime;
using ILGPU.Runtime.CPU;

static void TestILGPU_CPU()
{
    // Initialize ILGPU.
    Context context = Context.CreateDefault();
    Accelerator accelerator = context.CreateCPUAccelerator(0);
    
    // Load the data.
    var deviceData = accelerator.Allocate1D(new int[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 });
    var deviceOutput = accelerator.Allocate1D<int>(10_000);
    
    // load / compile the kernel
    var loadedKernel = accelerator.LoadAutoGroupedStreamKernel(
        (Index1D i, ArrayView<int> data, ArrayView<int> output) =>
        {
            output[i] = data[i % data.Length];
        });
    
    // tell the accelerator to start computing the kernel
    loadedKernel((int)deviceOutput.Length, deviceData.View, deviceOutput.View);
    
    // wait for the accelerator to be finished with whatever it's doing
    // in this case it just waits for the kernel to finish.
    accelerator.Synchronize();
    
    accelerator.Dispose();
    context.Dispose();
}

02 Memory and bandwidth and trheads

Computers need memory, and memory is slow.

GPU's have memory, and memory is slow.

https://mangkyu.tistory.com/85

NVIDIA가 만든 병렬 컴퓨팅 플랫폼 및 API 모델

GPGPU(General-Purpose Computing on Graphics Processing Unit)

CUDA Platform은 GPU의 가상 명령어셋과 병렬처리 요소들을 사용할 수 있도록 만들어주는 소프트웨어 레이어

Host: CPU and its memory (Host Memory)

Device: GPU and its memory (Device Memory)

1. Data copied from CPU to GPU

2. Launch VectorAdd kernel on the GPU

3. Resulting data copied from GPU to CPU

디바이스 코드는 nvcc (NVIDIA's Compiler)가 컴파일하는데, 몇가지 키워드가 필요함

__global__ 키워드는 Host를 통해 호출되어 Device에서 동작하는 함수

Block과 Thread

GPU 코드를 병렬로 처리하기 위한 단위

1개의 Block은 N개의 Thread로 구성됨

kernel <<< BlockCount, Threads-per-Block >>>(...)

#define N                   (2048 * 2048)
#define THREADS_PER_BLOCK   512

__global__ void dot(int *a, int *b, int *c)
{
    __shared__ int temp[THREADS_PER_BLOCK];
    int index = threadIdx.x + blockIdx.x * blockDim.x;
    temp[threadIdx.x] = a[index] * b[index];
    
    __syncthreads();
    
    if (threadIdx.x == 0)
    {
        int sum = 0;
        for (int i = 0; i < THREADS_PER_BLOCK; ++i)
            sum += temp[i];
        atomicAdd(c, sum);
    }
}

int main(void)
{
    int *p_a, *p_b, *p_c;
    int *p_dev_a, *p_dev_b, *p_dev_c;
    int cbSize = N * sizeof(int);
    
    // allocate host memories
    p_a = (int *)malloc(cbSize);
    p_b = (int *)malloc(cbSize);
    p_c = (int *)malloc(sizeof(int));
    
    // allocate device memories
    cudaMalloc(&p_dev_a, cbSize);
    cudaMalloc(&p_dev_b, cbSize);
    cudaMalloc(&p_dev_c, sizeof(int));
    
    // initialize variables
    for (int i = 0; i < N; ++i)
    {
        p_a[i] = i;
        p_b[i] = i;
    }
    
    // copy host memories to device memories
    cudaMemCpy(p_dev_a, p_a, cbSize, cudaMemcpyHostToDevice);
    cudaMemCpy(p_dev_b, p_b, cbSize, cudaMemcpyHostToDevice);
    
    // run dot with N threads
    dot<<< N / THREADS_PER_BLOCK, THREADS_PER_BLOCK >>>(p_dev_a, p_dev_b, p_dev_c);
    
    // copy device memories sum result(p_dev_c) to host memories(p_c)
    cudaMemCpy(p_c, p_dev_c, sizeof(int), cudaMemCpyDeviceToHost);
    
    printf("Total Sum: %d\n", *p_c);
    
    free(p_a);
    free(p_b);
    free(p_c);
    cudaFree(p_dev_a);
    cudaFree(p_dev_b);
    cudaFree(p_dev_c);
    
    return 0;
}

Atomic Operations

• atomicAdd()
• atomicSub()
• atomicMin()
• atomicMax()
• atomicInc()
• atomicDec()
• atomicExch()
• atomicCAS()

1.

// 1. Register HttpClientFactory
public void ConfigureServices(IServiceCollection services)
{
    // Other service registrations
    ;
    services.AddHttpClient();
    ;
    
    // Register your custom services
    services.AddScoped<IMyApiService, MyApiService>();
}

// 2. Create a Service Interface
public interface IMyApiService
{
    Task<string> GetApiResponseAsync();
}

// 3. Implement the Service Using HttpClientFactory
public class MyApiService : IMyApiService
{
    readonly IHttpClientFactory m_httpClientFactory;
    
    public MyApiService(IHttpClientFactory httpClientFactory)
    {
        m_httpClientFactory = httpClientFactory;
    }
    
    public async Task<string> GetApiResponseAsync()
    {
        // Create a named client or use the default client
        HttpClient client = m_httpClientFactory.CreateClient();
        
        // Make a HTTP GET request
        HttpResponseMessage response = await client.GetAsync("https://api.example.com/api/");
        
        if (response.IsSuccessStatusCode)
        {
            var sRespBody = await response.Content.ReadAsStringAsync();
            return sRespBody;
        }
        
        // Handle error
        return "Error occurred.";
    }
}

// Use the Service in a Controller
public class MyController : Controller
{
    readonly IMyApiService m_myApiService;
    
    public MyController(IMyApiService myApiService)
    {
        m_myApiService = myApiService;
    }
    
    public async Task<IActionResult> Index()
    {
        var sApiResponse = await m_myApiService.GetApiResponseAsync();
        return View(sApiResponse);
    }
}

2. Using Named HttpClient Instances

private static void ConfigureServices(IServiceCollection services)
{
    services.AddHttpClient("WebApiClient", config =>
    {
        config.BaseAddress = new Uri("https://localhost:5001/api/");
        config.Timeout = TimeSpan.FromSeconds(30);
        config.DefaultRequestHeaders.Clear();
    });
    
    ;
    services.AddScoped<IHttpClientFactoryService, HttpClientFactoryService>();
}

private async Task<List<TItem>> GetItemsWithHttpClientFactory<TItem>()
{
    var httpClient = m_httpClientFactory.CreateClient("WebApiClient");
    var response = await httpClient.GetAsync("items",
        HttpCompletionOption.ResponseHeadersRead);
    using (response)
    {
        response.EnsureSuccessStatusCode();
        
        var stream = await response.Content.ReadAsStreamAsync();
        var items = await JsonSerializer.DeserializeAsync<List<TItem>>(stream, m_jsopts);
        return items;
    }
}

3. Using Typed HttpClient Instances

// 1. Create the custom client
public class ItemsClient
{
    public HttpClient Client { get; }
    
    public ItemsClient(HttpClient client)
    {
        this.Client = client;
        this.Client.BaseAddress = new Uri("https://localhost:5001/api/");
        this.Client.Timeout = TimeSpan.FromSeconds(30);
        this.Client.DefaultRequestHeaders.Clear();
    }
}

// 2. Register
private static void ConfigureServices(IServiceCollection services)
{
    ;
    services.AddHttpClient<ItemsClient>();
    ;
}


public class HttpClientFactoryService : IHttpClientFactoryService
{
    readonly IHttpClientFactory m_httpClientFactory;
    readonly ItemsClient m_itemsClient;
    readonly JsonSerializerOptions m_jsopts;
    
    public HttpClientFactoryService(
        IHttpClientFactory httpClientFactory,
        ItemsClient itemsClients)
    {
        m_httpClientFactory = httpClientFactory;
        m_itemsClient = itemsClients;
        
        m_options = new JsonSerializerOptions { PropertyNameCaseInsentive = true };
    }
    
    private async Task<TItems> GetItemsWithTypedClient<TItem>()
    {
        var response = await m_itemsClient.Client.GetAsync("items",
            HttpCompletionOption.ResponseHeaderRead);
        using (response)
        {
            response.EnsureSuccessStatusCode();
            
            var stream = await response.Content.ReadAsStreamAsync();
            var items = await JsonSerializer.Deserialize<List<TItem>>(stream, m_jsopts);
            return items;
        }
    }
}

https://download.tek.com/document/37W_17249_6_Fundamentals_of_Real-Time_Spectrum_Analysis1.pdf

GLOSSARY

Acronym Reference