kgpu / kgpu / gpuops.cu
For setting up the buffer size of KGPU. Why not use dynamic programing to watch the video card that if the GPU is using shared memory with individual memory.
watch (RAM)
test(GPU memory)
lock (shared memory)
set max_kgpu_buf_size=(shared memory + gpu memory)/2
build first area(GPU memory)-->a1
build second area(share memory)--> a2
...
if (a1 not filled && flag!=1)
while (flag==2)
wait;
flag=0;
do assignment;
else
{
flag=1
reload current work;
assign to a2;
flag=2;
move less active work to shared memory.
}
This idea is to also use the assigned shared memory for any computation for later on. Also, as memory inside GPU is faster accessed by GPU itself, so put the memory of GPU at the first place, after it fills up, starts to use shared memory while do freeing GPU memory
2012年11月8日星期四
Look into Code
1.
/ services / test / sysbm / sysbm.c
/ services / test / sysbm / sysbm.c
#define MAX_MEM_SZ (32*1024*1024)
We can see that for current RAM, 32 GB is the upperbound.
tt = 1000000*(t1.tv_sec-t0.tv_sec) +
((long)(t1.tv_usec) - (long)(t0.tv_usec));
Is the 1000000 standing for 1024*1024??
2.
why not save error log under GPU's directory? Or even under /tmp or usr/tmp
Since this module will be working as part of kernel instead of user program, it is better to manage those logs together with system or GPU.
if (!ga) {
g_log(KGPU_LOG_ERROR, "mmap error\n");
kgpu_free_request(req);
free_page(TO_UL(buf));
return 0;
}
Here is to free the page file for empty/null/dealt request.
Why not collect those page files and accumulate to a large number to free them together at one time for higher performance?
ex.
create a queue
if (!ga)
add req to queue,
add to log
if (queue is full)
clear queue.
Here is to set default data/block size. Since this is for parallel computing why not use dynamic assignment with threshold that:static size_t decide_gpu_load(int disks, size_t dsize){if (dsize > (64*1024)) {return roundup(dsize/16, PAGE_SIZE);}return 0;}
if datasize> =threshold
roundup(dsize/16, PAGE_SIZE)
else roundup(dsize/4, PAGE_SIZE) // for smaller blocks
Try Boost with Designed and Tested Module
As the official website mentions that, it often requires manually loading the module and service for testing.
It is often using insmod to load modules and services.
So, in this way, is it possible to load those modules during boost time??
I have researched a lot of literature that nothing special says it is impossible, so let me try it first.
At first, make sure the grub version. It is better to have Grub 1.99 for Debian.
I tried:
grub rescue>set root=(hd0,5)
grub rescue>set prefix=(hd0,5)/grub
grub rescue>insmod /grub/normal.mod
grub rescue>insmod ./ecryptfs.ko
grub rescue>normal
Then after restarting,
sudo update-grub
sudo grub-install /dev/sda
Then I found that there is error that shows: unmounted sda: ecryptfs 1.
It is often using insmod to load modules and services.
So, in this way, is it possible to load those modules during boost time??
I have researched a lot of literature that nothing special says it is impossible, so let me try it first.
At first, make sure the grub version. It is better to have Grub 1.99 for Debian.
I tried:
grub rescue>set root=(hd0,5)
grub rescue>set prefix=(hd0,5)/grub
grub rescue>insmod /grub/normal.mod
grub rescue>insmod ./ecryptfs.ko
grub rescue>normal
Then after restarting,
sudo update-grub
sudo grub-install /dev/sda
Then I found that there is error that shows: unmounted sda: ecryptfs 1.
Several Issues and Questions
1. It is better to have SSD based file system for better performance.
Since almost all the operations within these codes rely on data transfer, and it is easy to be the bottleneck using traditional hard disk.
2. There are still non-tested modules, such as RAID 6.
Since it may require bigger number of hard disks to test RAID 6 recovery and R/W.
3. Here is a trick that, what happens to a GPU with dynamic shared memory?
As right now, almost no GPU only has shared memory, but at least has some G-RAM. This trick leads to problem of setting the parameters,
KGPU_BUF_SIZE.
In order to avoid unexpected result happening, make sure that this value is smaller than half of the video card memory.
Since almost all the operations within these codes rely on data transfer, and it is easy to be the bottleneck using traditional hard disk.
2. There are still non-tested modules, such as RAID 6.
Since it may require bigger number of hard disks to test RAID 6 recovery and R/W.
3. Here is a trick that, what happens to a GPU with dynamic shared memory?
As right now, almost no GPU only has shared memory, but at least has some G-RAM. This trick leads to problem of setting the parameters,
KGPU_BUF_SIZE.
In order to avoid unexpected result happening, make sure that this value is smaller than half of the video card memory.
2012年11月7日星期三
Relate Issues (and Possible Solution) 1
As CUDA 5.0 supports separable compilation, then I tried to upgrade it. However, it crashes. After i looked into the error log, I found that NVIDIA may not support CUDA 5.0, or the current working dirve does not support CUDA 5.0 well. Also, there is another option that, CUDA 4.2 may be good to upgrade.
After I did all the work that I did to set up the whole thing, there is error during run-time. So, I compile all the codes again, at this time it showed error message during compile time ( Device busy.). I searched the Internet, did not find any useful information about it. I can only assume that, the GPU I am using, NVIDIA 8200M does not support it.
There is nothing I can do but cange the version back to 4.0- or even 3.1.
Also, there is another issue, that when I choose another version of Linux kernel, such as 3.0.0.30, and 3.0.0.31, it failed to load ecryptfs module.
sudo modprobe ecryptfs is fine.
However, the code can still not complete their work.
The only thing I can find to make the code run further is to create a new private directory under usr.
After I did all the work that I did to set up the whole thing, there is error during run-time. So, I compile all the codes again, at this time it showed error message during compile time ( Device busy.). I searched the Internet, did not find any useful information about it. I can only assume that, the GPU I am using, NVIDIA 8200M does not support it.
There is nothing I can do but cange the version back to 4.0- or even 3.1.
Also, there is another issue, that when I choose another version of Linux kernel, such as 3.0.0.30, and 3.0.0.31, it failed to load ecryptfs module.
sudo modprobe ecryptfs is fine.
However, the code can still not complete their work.
The only thing I can find to make the code run further is to create a new private directory under usr.
Second Test: eCryptfs
This test is to test on the throughput on eCryptfs based on RAM
Block Size CPU GPU Operation RAM GRAM
4 KB 80.5MB/s 288MB/s READ DDR2 GDDR3
16KB 93MB/s 395MB/s READ DDR2 GDDR3
64KB 85MB/s 321MB/s READ DDR2 GDDR3
4 KB 35MB/s 35MB/s WRITE DDR2 GDDR3
16KB 37MB/s 40MB/s WRITE DDR2 GDDR3
64KB 37MB/s 43MB/s WRITE DDR2 GDDR3
The GPU can work with the given code with minor change on settings.
Also, from the result, as the block size is becoming larger, the performance of GPU is better, however, the performance under WRITE is not increasing significantly.
Block Size CPU GPU Operation RAM GRAM
4 KB 80.5MB/s 288MB/s READ DDR2 GDDR3
16KB 93MB/s 395MB/s READ DDR2 GDDR3
64KB 85MB/s 321MB/s READ DDR2 GDDR3
4 KB 35MB/s 35MB/s WRITE DDR2 GDDR3
16KB 37MB/s 40MB/s WRITE DDR2 GDDR3
64KB 37MB/s 43MB/s WRITE DDR2 GDDR3
The GPU can work with the given code with minor change on settings.
Also, from the result, as the block size is becoming larger, the performance of GPU is better, however, the performance under WRITE is not increasing significantly.
First Test: dm-crypt
This test is to test the read speed on GPU reading file on different given block size:
Block Size CPU GPU HDD RAM GRAM
4 KB 20MB/s 30MB/s 5400rpm DDR2 GDDR3
16KB 23MB/s 37MB/s 5400rpm DDR2 GDDR3
64KB 29MB/s 38.5MB/s 5400rpm DDR2 GDDR3
4 KB 21MB/s 29MB/s 7200rpm DDR2 GDDR3
16KB 24.5MB/s 40MB/s 7200rpm DDR2 GDDR3
64KB 31MB/s 39MB/s 7200rpm DDR2 GDDR3
From this result, GPU has a better performance than CPU, however, the HDD has little influence on the performance. However, the main memory and memory of GPU may be the bottleneck.
Also, as
KGPU_BUF_SIZE*2 < Size of Your Video
I set it to 200 in
kgpu/kgpu.h
订阅：
博文 (Atom)