Look into Code 2

   kgpu / kgpu / gpuops.cu

For setting up the buffer size of KGPU. Why not use dynamic programing to watch the video card that if the GPU is using shared memory with individual memory.

watch (RAM)
test(GPU memory)
lock (shared memory)
set max_kgpu_buf_size=(shared memory + gpu memory)/2
build first area(GPU memory)-->a1
build second area(share memory)--> a2
...
if (a1 not filled && flag!=1)
   while (flag==2)
   wait;
   flag=0;
  do assignment;
else 
{
    flag=1
    reload current work;
    assign to a2;
    flag=2;
    move less active work to shared memory.
   
} 

This idea is to also use the assigned shared memory for any computation for later on. Also, as memory inside GPU is faster accessed by GPU itself, so put the memory of GPU at the first place, after it fills up, starts to use shared memory while do freeing GPU memory

Look into Code

1.
/ services / test / sysbm / sysbm.c

#define MAX_MEM_SZ (32*1024*1024)

We can see that for current RAM, 32 GB is the upperbound.

tt = 1000000*(t1.tv_sec-t0.tv_sec) +

((long)(t1.tv_usec) - (long)(t0.tv_usec));

 

Is the 1000000 standing for 1024*1024??

 

2.

kgpu / services / test / callgpu / callgpu.c

 

why not save error log under GPU's directory? Or even under /tmp or usr/tmp 

Since this module will be working as part of kernel instead of user program, it is better to manage those logs together with system or GPU.

 

 

3. kgpu / services / test / calg2 / calg2.c

   if (!ga) {

   g_log(KGPU_LOG_ERROR, "mmap error\n");

   kgpu_free_request(req);

   free_page(TO_UL(buf));

   return 0;

    }

 

Here is to free the page file for empty/null/dealt request.

Why not collect those page files and accumulate to a large number to free them together at one time for higher performance?

 

ex.

 

create a queue

if (!ga)

add req to queue,

add to log

 

if (queue is full)

clear queue.

 

 

 

 

4. / services / raid6 / gpq / gpq.c

  

static size_t decide_gpu_load(int disks, size_t dsize)

{

    if (dsize > (64*1024)) {

 return roundup(dsize/16, PAGE_SIZE);

    }

    return 0;

}

 Here is to set default data/block size. Since this is for parallel computing why not use dynamic assignment with threshold that:

 

if  datasize> =threshold

 

roundup(dsize/16, PAGE_SIZE)

else  
roundup(dsize/4, PAGE_SIZE) // for smaller blocks

  

Try Boost with Designed and Tested Module

As the official website mentions that, it often requires manually loading the module and service for testing.
It is often using insmod to load modules and services.
So, in this way, is it possible to load those modules during boost time??
I have researched a lot of literature that nothing special says it is impossible, so let me try it first.

At first, make sure the grub version. It is better to have Grub 1.99 for Debian.
I tried:
grub rescue>set root=(hd0,5)
grub rescue>set prefix=(hd0,5)/grub
grub rescue>insmod /grub/normal.mod
grub rescue>insmod  ./ecryptfs.ko
grub rescue>normal
Then after  restarting,
  sudo update-grub

    sudo grub-install /dev/sda

Then I found that there is error that shows: unmounted sda: ecryptfs 1.

Several Issues and Questions

1. It is better to have SSD based file system for better performance.
  Since almost all the operations within these codes rely on data transfer, and it is easy to be the bottleneck using traditional hard disk.

2. There are still non-tested modules, such as RAID 6.
 Since it may require bigger number of hard disks to test RAID 6 recovery and R/W.

3. Here is a trick that, what happens to a GPU with dynamic shared memory?
As right now, almost no GPU only has shared memory, but at least has some G-RAM. This trick leads to problem of setting the parameters,
KGPU_BUF_SIZE.
In order to avoid unexpected result happening, make sure that this value is smaller than half of the video card memory.

Relate Issues (and Possible Solution) 1

As  CUDA 5.0 supports separable compilation, then I tried to upgrade it. However, it crashes. After i looked into the error log, I found that NVIDIA may not support CUDA 5.0, or the current working dirve does not support CUDA 5.0 well. Also, there is another option that, CUDA 4.2 may be good to upgrade. 
After I did all the work that I did to set up the whole thing, there is error during run-time. So, I compile all the codes again, at this time it showed error message during compile time ( Device busy.). I searched the Internet, did not find any useful information about it. I can only assume that,  the GPU I am using, NVIDIA 8200M does not support it. 
There is nothing I can do but cange the version back to 4.0- or even 3.1.

Also, there is another issue, that when I choose another version of Linux kernel, such as 3.0.0.30, and 3.0.0.31, it failed to load ecryptfs module. 

sudo modprobe ecryptfs is fine.
However, the code can still not complete their work.
The only thing I can find to make the code run further is to create a new private directory under usr.

Second Test: eCryptfs

This test is to test on the throughput on  eCryptfs based on RAM




Block Size           CPU              GPU        Operation             RAM           GRAM
4 KB                    80.5MB/s      288MB/s     READ                DDR2          GDDR3
16KB                   93MB/s         395MB/s     READ                DDR2           GDDR3
64KB                   85MB/s         321MB/s     READ               DDR2          GDDR3
4 KB                    35MB/s         35MB/s      WRITE                DDR2           GDDR3
16KB                   37MB/s         40MB/s       WRITE                DDR2          GDDR3
64KB                    37MB/s        43MB/s       WRITE              DDR2          GDDR3

The GPU can work with the given code with minor change on settings.
Also, from the result, as the block size is becoming larger, the performance of GPU is better, however,  the performance under WRITE is not increasing significantly.

First Test: dm-crypt

This test is to test the read speed on GPU reading file on different given block size:

Block Size           CPU              GPU         HDD                       RAM           GRAM
4 KB                    20MB/s         30MB/s     5400rpm                DDR2          GDDR3
16KB                   23MB/s         37MB/s     5400rpm                DDR2           GDDR3
64KB                   29MB/s         38.5MB/s   5400rpm               DDR2          GDDR3
4 KB                    21MB/s         29MB/s     7200rpm                DDR2           GDDR3
16KB                   24.5MB/s      40MB/s     7200rpm                DDR2          GDDR3
64KB                   31MB/s         39MB/s      7200rpm               DDR2          GDDR3



From this result, GPU has a better performance than CPU, however, the HDD has little influence on the performance.  However, the main memory and memory of GPU may be the bottleneck.
Also, as 
KGPU_BUF_SIZE*2 < Size of Your Video
I set it to 200 in
kgpu/kgpu.h

Building up CUDA 2

To install the SDK:

Enter the directory of SDK.
cd ~/NVIDIA_GPU_Computing_SDK/C

Then run the makefile.

As there are a lot of packages undefined or needed, I researched for the solution, there are the steps I did:

% install glu.h
sudo apt-get install freeglut3-dev

% install g++ 4.3
sudo apt-get install g++-4.3

% figure out IGL

sudo rm /usr/lib/libGL.so

cd /usr/lib 

sudo ln -s libGL.so.304.10 libGL.so 

    -release :     input "make"
    -debug   :     input "make dbg=1"
    -emurelease:    input "make emu=1"
    -emudebug:    input "make emu=1 dbg=1" 

http://www.bfcat.com/index.php/blog_/

  

Building up CUDA 1

I downloaded and run the following:
sudo sh cudatoolkit_3.1_linux_32_ubuntu9.10.run
install  it to  /usr/local/cuda
install:  sh gpucomputingsdk_3.1_linux.run
to ~/NVIDIA_GPU_Computing_SDK
setting:
sudo gedit ~/.bashrc
add 
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH

As CUDA is complied by GCC 4.3, so I downloaded :
gcc-3.4-base_3.4.6-6ubuntu3_i386.deb、
gcc-3.4_3.4.6-6ubuntu3_i386.deb、
cpp-3.4_3.4.6-6ubuntu3_i386.deb、
g++-3.4_3.4.6-6ubuntu3_i386.deb、
libstdc++6-dev_3.4.6-6ubuntu3_i386.deb

and installed them.
Then:
sudo apt-get install g++-4.3 gcc-4.3

As multiple versions of Gcc are needed:

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 44 --slave /usr/bin/g++ g++ /usr/bin/g++-4.4 --slave /usr/bin/gcov gcov /usr/bin/gcov-4.4

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 43 --slave /usr/bin/g++ g++ /usr/bin/g++-4.3 --slave /usr/bin/gcov gcov /usr/bin/gcov-4.3

After that:

sudo update-alternatives --display gcc

For changing version of Gcc:

sudo update-alternatives --config gcc 
change the version to 4.3  

GPU Drive

As there is default GPU drive coming with Ubuntu, I deleted those under the folder:
/lib/linux-restricted-modules/2.6.22-14-generic/
and manually installed the drives in my list one by one.
And after a few rounds of testing, the result is as following:

Version:                                   Compatibility:
300                                             good
304                                             good
310 test                                      not working
310 beta                                     not working
x.org                                          can enter G3, only acceptable


I choose version 304 as my final choice.

Building up Basic Infras.

As it mentions in officia website of KGPU and CUDA,  NVIDIA GeForce 8200M is the lowest GPU that can support CUDA's features, and Linux with certian Linux kernel is tested under this project.
The following is my list:

GPU                      Cores                      Bandwidth              RAM
8200MG                 8                                 128bit                GDDR 3
9600M GT              32                               128bit                GDDR 3

http://baike.baidu.com/view/4395575.htm
http://baike.baidu.com/view/4394615.htm

Also, it mentions that it may only work under 64-bit Linux , that is

2.6.39.4 x86_64

I downloaded Ubuntu Linux and installed a few versions, just want to find out the exact same 

version of Linux kernel. As a result, Ubuntu 11.04 and 10.12 have the working version of 2.6.39.4.

Also, I made a backup of the Linux kernel for later use.

I also downloaded the open source and official NVIDIA drivers for Linux system. Here is the list:

 version 304

 version 310 test

 version 300

 version 310 beta1-ubuntu

  x.org  open source drive

I will test those drives within a few days.

CUDA Knowledge

I have spent this week and most of last week studying on CUDA, including the basic knowledge of architecture and programming issues.
This includes the basic knowledge of nvcc compiler-- how it works and how to handle and utilize it under Linux OS, and how to build up a IDE for CUDA C programming.  Likely, what new features it has and what new support it provides under Linux. In addition, other CUDA platform may also help in case when official CUDA may not perfectly work which may be CUDA light and/or Open CUDA. 

http://baike.baidu.com/view/1488597.htm
www.nvidia.ca/object/cuda_home_new.htm
http://code.google.com/p/kgpu
https://github.com/wbsun/kgpu

Hardware Support

Some messages on the project official website inquire that whether those released codes can work with older NVIDIA GPUs. From my personal knowledge and research that the least GPU can support CUDA 4.0 and the KGPU released code is NVIDIA GeForce 8200M, which I want to use for this project, and also Geforce 9600M GT as backup, if there is something going wrong with the first GPU. Also, the aim of this project is to build up the parallelism on GPU and CPU, so the more cores GPU has, the better result may have derived from it.

Software and System Pre-analysis

 The main idea of the project is to invoke the threads created by GPU to work the the pieces of work from CPU. Since the project is open source project, and a lot of researchers have tried to introduce the knowledge to ATI and Intel GPU/On-board Chips, however, it is less likely to achieve presently. It seems that it is still  best to achieve goals and research  approaches with NVIDIA hardware.
On the other hand, as Windows and Mac OS really care about their "security", Linux is the best and only system supports the project. Also, it is also said that KGPU is to help with accelerating Linux kernel. In most of the found literatures, authors use a fixed Linux kernel, 2.6.4.39. With newer techniques inbuilt, 3.0 plus version of Linux kernel may provide better support for this project.

Backgroud and Importance of KGPU

As CUDA develops to version 4.0 and plus, NVIDIA started to hold a open source project to focus on building up parallelism of CPU and NVIDIA GPU, that is by utilizing the platform of CUDA, it is possible for GPU to work on kernel--called KGPU. Theoretically, the computation output is easily to become the bottleneck when the workload is fixed. In another word, GPU can work as GPGPU (General Purpose GPU) to accelerate the kernel. This project is very attractive that, so far, the parallel computing is built up within CPU or even between CPU and the main memory, once GPU can do the work of CPU as required, the new era of parallel computing comes.