Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SvdModel2LstmSDSoCV2-no-unroll #1

Open
wants to merge 71 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
9e2de2d
Storage class compiles. Testing call to SvdModel2LstmSDSoCV2() accele…
ribesstefano Jun 30, 2021
cebc6de
Call to SvdModel2LstmSDSoCV2() accelerator completes w/out hanging (n…
ribesstefano Jun 30, 2021
9211e04
Added call to software model. (There might still be a segmentation-fa…
ribesstefano Jul 2, 2021
8836595
Added call to HLS emulator software. (There might still be a segmenta…
ribesstefano Jul 5, 2021
b30eab4
Added check on accelerator output and HLS emulator
ribesstefano Jul 6, 2021
1779240
Started working on synthesizing the design.
ribesstefano Jul 11, 2021
adabfe7
Added AXIS lib. Synthesis for SvdModel2LstmSDSoCV2 works.
ribesstefano Jul 14, 2021
02e2eec
Renamed hls_utils namespace to hlsutils
ribesstefano Jul 15, 2021
d23301c
Started working on standalone U-Unit (HlsKernelU). Removed relax_ii_f…
ribesstefano Jul 15, 2021
ba22e19
Added CMake support for Vitis.
ribesstefano Jul 16, 2021
9fb1525
Added KernelU test and started playing with hls::vector.
ribesstefano Jul 16, 2021
caf86f3
Written vectorized U-Kernel.
ribesstefano Jul 17, 2021
0bf93bd
Adjusted scaling of testing values. Still testing HlsVectorKernelU co…
ribesstefano Jul 17, 2021
0f516f3
Fixed bug in HlsVectorKernelU.
ribesstefano Jul 18, 2021
1bed5f7
Co-Simulation of HlsVectorKernelU completes. There's still one M_AXI …
ribesstefano Jul 19, 2021
b8b637d
Added hls::vector to AXIS lib. Added stable version of Scaled SVD app…
ribesstefano Jul 21, 2021
23d2051
Fixed PopVector bug in axis_lib
ribesstefano Jul 21, 2021
7cc1879
AxiStreamInterface class now synthesizes in Vitis and supports Vitis …
ribesstefano Jul 21, 2021
afc293d
Fixed bugs regarding hls::vectors in axis_lib
ribesstefano Jul 22, 2021
183349f
KernelU: increased width of output stream. Added PYNQ directory for s…
ribesstefano Jul 24, 2021
bea5c83
Asserted TKEEP and TSTRB always HIGH in AxisInterface wrapper. FIFOs …
ribesstefano Jul 27, 2021
ada8166
Added generic Matrix-Vector multiplier kernel.
ribesstefano Jul 30, 2021
7250d6e
Started working on KernelU with different sampling rates.
ribesstefano Aug 2, 2021
ce3fa0a
Written input dispatcher for KernelU with different refinements
ribesstefano Aug 2, 2021
9cf4a61
Added working KernelU with different samplings. Starting optimizing.
ribesstefano Aug 3, 2021
3acd5d8
Starting simplifying KernelU with different refinements.
ribesstefano Aug 4, 2021
a3ceb6f
Kernel HlsKernelU_ManySampling synthesizes with II=1. Testing still n…
ribesstefano Aug 6, 2021
dc982a0
HlsKernelU_ManySampling: fixed TLAST bug.
ribesstefano Aug 6, 2021
44d6867
KernelU: Fixed results for N == 2. Bug still present for N > 2.
ribesstefano Aug 7, 2021
ac51a32
KernelU: Fixed bug in calculating R_total.
ribesstefano Aug 8, 2021
97a61b4
Added generic KernelS and initial version of KernelV.
ribesstefano Aug 8, 2021
f79ad97
Started working on generic input size for KernelU and KernelV.
ribesstefano Aug 8, 2021
d45bff0
KernelU: flexible input size works in simulation.
ribesstefano Aug 14, 2021
6191501
KernelU: flexible number of inputs works in simulation.
ribesstefano Aug 14, 2021
49aad39
KernelU: moved to a templated design.
ribesstefano Aug 20, 2021
7060615
Added templated and flexible versions of KernelU, KernelS and KernelV.
ribesstefano Aug 21, 2021
5ec1107
Added Policy-based AXI stream interface in order to customize kernel …
ribesstefano Aug 24, 2021
0c7aad0
HlsSvdKernel synthesis now proceeds.
ribesstefano Aug 24, 2021
b84b987
Added compile flag to truncate long HLS names. HlsSvdKernel synthesis…
ribesstefano Aug 24, 2021
69d1def
Renamed packet types in AXIS policy classes. Converted x_buffer in Ke…
ribesstefano Aug 25, 2021
a04b7b0
Made SvdKernel output port generic to be included in LSTM kernel.
ribesstefano Aug 25, 2021
05c3bc8
Moved LSTM and Dense files into a separate 'layers/' folder.
ribesstefano Aug 26, 2021
a7019d8
Added DenseSvdKernel.
ribesstefano Aug 26, 2021
f47ebf5
Added test for KernelV.
ribesstefano Aug 26, 2021
513c095
KernelV: correct simulation for single input.
ribesstefano Aug 27, 2021
b0a62b9
KernelV: fixed bug with processing multiple inputs.
ribesstefano Aug 27, 2021
948878f
KernelV: fixed bug with processing multiple inputs.
ribesstefano Aug 27, 2021
783aa4c
KernelV: re-arrenged loops order.
ribesstefano Aug 30, 2021
0fce07b
Setup Pynq directory and added PYNQ kernel_u project.
ribesstefano Sep 1, 2021
3881708
Added kernel_v bitstream to PYNQ projects.
ribesstefano Sep 1, 2021
1b3abd2
PYNQ/kernel_v: Working hardware with small dimensions
ribesstefano Sep 1, 2021
4f36469
KernelV: fixed cosimulation. Starting moving to Vitis 2021.1 (can't a…
ribesstefano Sep 4, 2021
05e855c
Made kernels compatible with Vitis 2021.1. Added overlay files to PYN…
ribesstefano Sep 5, 2021
52bd28f
HlsDenseSvd and HlsLstmSvd are synthesizeable in Vitis 2021.1.
ribesstefano Sep 5, 2021
a0f3db4
Tested HlsKernelV in PYNQ.
ribesstefano Sep 5, 2021
04a9188
AxisLib: added method to push a buffer into a single packet. HlsLstmS…
ribesstefano Sep 5, 2021
c2da42e
PYNQ: Added loading and allocating from binary file into buffer (exam…
ribesstefano Sep 5, 2021
b158bb8
Added Cosimulation-passing HlsDenseSvd design. Added HlsLstmSvd design.
ribesstefano Sep 7, 2021
b486ae3
Started working on wrapper function for HlsLstmSvd.
ribesstefano Sep 7, 2021
ab2948e
HlsDenseSvd: fixed bug in generating TLAST signal for output port.
ribesstefano Sep 7, 2021
4dba96e
HlsDenseSvd: Added PYNQ hardware files.
ribesstefano Sep 7, 2021
62aa779
HlsSvdKernel: added PYNQ files.
ribesstefano Sep 8, 2021
15f87ae
HlsDenseSvd: updated PYNQ hardware files.
ribesstefano Sep 8, 2021
4610a19
PYNQ: added notebooks on HlsDenseSvd and HlsSvdKernel. HlsDenseSvd wo…
ribesstefano Sep 8, 2021
347e58f
HlsLstmSvd: added testbench.
ribesstefano Sep 11, 2021
fce3883
HlsSvdKernel: added testbench.
ribesstefano Sep 11, 2021
215c3ee
Changed pipeline style to free-running/flushing.
ribesstefano Sep 11, 2021
c1183b8
HlsSvdKernel: updated PYNQ bitstream file.
ribesstefano Sep 12, 2021
dd79871
SvdModel2LstmSDSoCV2: started clean-up and made synthesizeable in Vit…
ribesstefano Sep 16, 2021
20175dd
SvdModel2LstmSDSoCV2: removed unroll pragmas in top function, gate di…
ribesstefano Sep 16, 2021
41d2ec0
SvdModel2LstmSDSoCV2: removed all unrolling
ribesstefano Sep 16, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,12 @@ data/game_*
data/windows*
# Hardware Directories
vivado_hls.log
./hls/
vitis_hls.log
hls_prj/
vivado/
vitis_include/
./token

# PYNQ
**/.ipynb_checkpoints
**/sds_trace_data.dat
51 changes: 34 additions & 17 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,36 +1,53 @@
cmake_minimum_required(VERSION 3.10)
# Set the project name
project(Svd VERSION 1.0)
# Specify the C++ standard
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED True)

# To locate "custom"/manually added libraries
# To locate "custom", i.e. manually added, libraries
list(APPEND CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/cmake/Modules)

# Locate libraries and headers (see Files in ./cmake/Modules/)
# find_package(Vitis REQUIRED)
find_package(Vivado REQUIRED)
find_package(OpenCv REQUIRED)
# Locate external libraries and headers (see Files in ./cmake/Modules/)
# Search for HLS: if Vitis is found, use C++14, else fall back to C++11.
find_package(Vitis REQUIRED)
if (Vitis_FOUND)
# Specify the C++14 standard
message("[INFO] Vitis HLS FOUND.")
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(HLS_INCLUDE_DIRS ${VITIS_INCLUDE_DIRS})
add_compile_definitions(__VITIS_HLS__)
else()
message("[INFO] Vivado HLS FOUND.")
find_package(Vivado REQUIRED)
# Specify the C++11 standard
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(HLS_INCLUDE_DIRS ${VIVADO_INCLUDE_DIRS})
endif()
# find_package(OpenCv REQUIRED)

# set(HLS_INCLUDE_DIRS ${VITIS_INCLUDE_DIRS})
# set(HLS_INCLUDE_DIRS ${VIVADO_INCLUDE_DIRS})
# message(${HLS_INCLUDE_DIRS})
message(${HLS_INCLUDE_DIRS})

# Add all definitions
if (WIN32)
add_compile_definitions(IMAGE_OUTPUT_PATH="C:/Users/ste/phd/hls_projects/hls_svd/data")
else()
add_compile_definitions(IMAGE_OUTPUT_PATH="/mnt/c/Users/ste/phd/hls_projects/hls_svd/data")
endif()
# The following definitions is required for compiling half-precision numbers.
add_compile_definitions(HLS_NO_XIL_FPO_LIB)
# add_compile_definitions(USE_FLOAT)
add_compile_definitions(DEBUG_LEVEL=2)

add_compile_definitions(INPUT_SIZE=1024)
add_compile_definitions(HIDDEN_SIZE=512)
add_compile_definitions(NUM_GATES=4)
add_compile_definitions(NUM_SAMPLES=2)
add_compile_definitions(NUM_TILES_U=4)
add_compile_definitions(NUM_ZERO_TILES_U=1)
add_compile_definitions(NUM_TILES_V=4)
add_compile_definitions(NUM_ZERO_TILES_V=1)
add_compile_definitions(NUM_TIMESTEPS=28)
add_compile_definitions(FIX_WIDTH=16)
add_compile_definitions(FIX_FRACT_WIDTH=5)

# Move executable in bin/, along side the DLLs (copied)
set(EXECUTABLE_OUTPUT_PATH ${PROJECT_SOURCE_DIR}/bin)
file(COPY ${OpenCv_LIBS} DESTINATION ${EXECUTABLE_OUTPUT_PATH})
# file(COPY ${OpenCv_LIBS} DESTINATION ${EXECUTABLE_OUTPUT_PATH})

# NOTE: an object file becomes a library. All libraries/objects must be LINKED later!
# Tell the application where to find the other CMake config files.
Expand Down
105 changes: 98 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,108 @@ cmake ..
make all
```

## Notes on Using Vitis

### AXIS Interface and DMA

Vitis to include the TLAST side channel if and only if TKEEP and TSTRB are also included.

In order to attach the port to a Xilinx DMA, the TLAST signal must be properly set HIGH at the end of the data transmission.

The TKEEP and TSTRB signals must be *always* set to HIGH, as indicated in the [AXIS documentation](https://developer.arm.com/documentation/ihi0051/a/Interface-Signals/Byte-qualifiers/TKEEP-and-TSTRB-combinations).


### Partitioning hls::vector Arrays

A standard way of partitioning an array is:
```c++
hls::stream<hls::vector<int, 4> > x_streams[M][N];
#pragma HLS ARRAY_PARTITION variable=x_streams complete dim=0
```
However, since we are dealing with a `hls::vector` type, setting `dim=0` (all dimensions) will partition the array on the vector dimension too.

In the example above, Vitis will create `M * N * 4` different streams (instead of just `M * N`). To fix it, manually specify the partitioning on the dimensions, like so:
```c++
hls::stream<hls::vector<int, 4> > x_streams[M][N];
#pragma HLS ARRAY_PARTITION variable=x_streams complete dim=1
#pragma HLS ARRAY_PARTITION variable=x_streams complete dim=2
```

### Implementing AXIS Interfaces

In order to implement AXIS interfaces, avoid using `depth` in the pragma, as follows:
```c++
const int kAxiBitwidth = 128;

void HlsVectorKernelU(hls::stream<ap_axiu<kAxiBitwidth, 0, 0, 0> >& x_port,
hls::stream<ap_axiu<kAxiBitwidth, 0, 0, 0> >& y_port) {
#pragma HLS INTERFACE axis port=x_port // depth=... <- DON'T SPECIFY THE DEPTH!
#pragma HLS INTERFACE axis port=y_port // depth=... <- DON'T SPECIFY THE DEPTH!
// ...
}
```
The type `ap_axiu` must now be used to generate AXIS with side channels. Note: for using external DMAs, we need the TLAST, TKEEP and TSTRB signals. In particular, TKEEP and TSTRB must be all set (i.e. all ones) in order to signal data packets.

#### AxiStreamInterface Class

This repository contains a wrapper class for kernel arguments of type `hls::stream` named `AxiStreamInterface`. The class is implemented following a _Policy-based_ C++ paradigm, meaning that it accepts either a `AxiStreamPort` or `AxiStreamFifo` as possible policies (in practice, a template argument).

The idea is to have a kernel argument, i.e. an HLS port, which can be either an AXIS interface with side-channels, or a bare FIFO interface connected to another kernel. In fact, Vitis HLS doesn't allow stream interfaces with side-channels within an IP. To overcome the issue, the `AxiStreamInterface` can be customized to be an IP port or a FIFO port, depending on the use of the kernel.

An example of this can be seen in `HlsKernelU` and in `svd::SvdKernel`, which specialize the `svd::KernelU` function template. In the first case, the `svd::KernelU` has its output stream port `xu_port` connected to one of the IP's ports (with side-channels). In the latter case instead, `svd::KernelU` is connected to `svd::KernelS`, and so its `xu_port` argument is an internal FIFO (without side-channels).

The `AxiStreamInterface` class in `axis_lib.h` can also be used with `hls::vector` types.

### HLS Vector Patch

If the project will be compiled with the Vitis HLS libraries, it needs a patch in the `hls::vector` class.

Simply add the following line in the `vector` class after the `public:` statement:
```c++
public:
static const int width = N;
```

In this way, one can access the number of elements in a `hls::vector` at compile/synthesis time by doing:

```c++
hls::vector<int, 5> a;
std::cout << "Number of elements in a: " << a::width << std::endl;

// > Number of elements in a: 5
```

## Notes on PYNQ Design

### Vivado Project

#### Xilinx DMA

The DMA should be configured in the following way:

* Max burst length to maximum
* Register buffer width to maximum

#### HP Ports

All HP ports should be set to 64bit width (to avoid receiving data interleaved by zeroes).


## TODOs

List of TODOs:
* Import u, s, v new kernels
* Import (and clean up?) u, s, v old kernels
* Import DMA functions
* Import and clean up HLS SVD-model-Bouganis
* Import and clean up HLS SVD-model-2LSTM
* Import some testbenches to try compile something

* ~Import u, s, v new kernels~
* ~Import (and clean up?) u, s, v old kernels~
* ~Import DMA functions~
* ~Import and clean up HLS SVD-model-Bouganis~
* ~Import and clean up HLS SVD-model-2LSTM~
* ~Import some testbenches to try compile something~

## Bugs

List of possible bugs:
* Having not squared images in games generates distorted images.

* Constructing data handler storage might lead to segmentation faults.
* Having `R == 1` might trigger some asserts.
* Having `output_size == H` in HlsKernelV might break hardware runs.
8 changes: 5 additions & 3 deletions cmake/Modules/FindVitis.cmake
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
if (WIN32)
set(VITIS_INCLUDE_DIRS D:/Programs/Xilinx/Vitis_HLS/2020.2/include/)
# set(VITIS_INCLUDE_DIRS D:/Programs/Xilinx/Vitis_HLS/2021.1/include/)
set(VITIS_INCLUDE_DIRS C:/Users/ste/phd/hls_projects/hls_svd/vitis_include/2020.2/include/)
# set(VITIS_INCLUDE_DIRS C:/Users/ste/phd/hls_projects/hls_svd/vitis_include/2021.1/include/)
else()
set(VITIS_INCLUDE_DIRS /mnt/d/Programs/Xilinx/Vitis_HLS/2020.2/include/)
set(VITIS_INCLUDE_DIRS /mnt/d/Programs/Xilinx/Vitis_HLS/2021.1/include/)
endif()

# NOTE: It handles the REQUIRED, QUIET and version-related arguments of find_package.
# It also sets the <PackageName>_FOUND variable. The package is considered found
# if all variables listed contain valid results, e.g. valid filepaths.
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Vitis DEFAULT_MSG VITIS_INCLUDE_DIRS)
find_package_handle_standard_args(Vitis DEFAULT_MSG VITIS_INCLUDE_DIRS)
Loading