v3.7.3

This is based off of v3.7.2 tag, with a backported DAQ input processor bugfix from v3.8.6, which fixed a potential for VFAT packets to go out of sync due to bad VFAT word counting in the case when new data arrives exactly in the clock cycle when the timeout flag is set. The result of this is seen as VFAT EC/BC mismatches, where at some point a VFAT packet from a previous event would show up in all events, with no signs of errors.

v3.8.4

TTC calibration mode added, see below for more details.
Added VFAT-VFAT mixed BC and mixed EC flags in the data format
Removed ADC monitoring from the SCA controller
SCA TTC_HARD_RESET_EN is now an OH mask instead of just one bit
BC now starts at 1 instead of 0 to match VFAT BC counting
Added a possibility to switch to 40MHz promless programming mode

======== TTC calibration mode ========
In this mode, CTP7 will generate a calpulse followed by L1A upon receipt of L1A signal from AMC13. This can be useful since AMC13 has some rate limitations for calpulsing, but not for L1As.
This is controlled with two registers:
GEM_AMC.TTC.CTRL.CALIBRATION_MODE: Enables calibration mode when set to 1. In calibration mode the incoming L1A commands will actually trigger a calpulse signal to the frontend, followed by an L1A after a fixed delay, which is controlled by the CALPULSE_L1A_DELAY register below.
GEM_AMC.TTC.CTRL.CALPULSE_L1A_DELAY: When calibration mode is enabled, this register controls the delay between the calpulse and the L1A to the frontend e.g. a setting of 1 will have the calpulse and L1A sent in consecutive clock cycles, a setting of 2 will have one empty BX between the calpulse and L1a, etc.. Note that 0 is not a valid setting and will result in no L1As being sent to frontend. Also note that if another L1A comes in while this delay is still being processed, it will restart the delay and result in the previous L1A to the frontend being cancelled, so don't set this delay longer than the delay between consecutive L1As in AMC13.

v3.8.2

v3.8.1 -- Bugfix in the OH FPGA communication FSM

v3.8.2 -- Fixed not-in-table and disperr counters, introduced MGT registers to the address table. This is first compile for GE1/1 with new OH comm protocol and trigger link protocol with BC0 and resync symbols

v3.8.0

Switched to the new OH FPGA communication protocol from Andrew, which uses 6b8b encoding and only one elink. Also added some GE2/1 support.

Currently only GE2/1 release is available.

v3.7.2

Fixed a bug related to miniPOD links -- some config registers were missing, preventing the links from starting up correctly. This means that in v3.7.0 and v3.7.1 the trigger links were not working at all.

Dummy trigger (EMTF) outputs added on miniPOD TX for easy loopback testing of the miniPOD trigger inputs. This is also a foundation for the EMTF output, only need to implement the protocol now.

v3.7.1

ADC0 and ADC1 registers have been renamed to ADC0_CACHED and ADC1_CACHED, which are now returning a value from CTP7 cache instead of the VFAT itself (this is faster and will not trigger a timeout in Zynq). To update the cache the user has to read or write ADC0_UPDATE or ADC1_UPDATE registers (this will take up to 20us to update the cache)

v3.7.0

Introduced MiniPOD links, and moved trigger inputs there, allowing to expand the number of OHs to 12 (see fiber mapping at the end of this post).

Added a config blaster interface. This module is used to store the front-end hardware configuration, which is then streamed automatically to all front-end devices after each hard-reset.

This version implements the configuration storage and interface to the software, but the actual configuration streaming to the hardware is not yet implemented.

There are 3 RAMs that software can write and read: one for GBTXs, one for VFAT3s, and one for OH FPGAs. Each of these RAMs are given 16 bits of address space (2Mbits), but the actual RAMs in the firmware are sized according to the number of OHs supported by that firmware build, thus software should only write the configuration of the supported number of devices (otherwise it may go out-of-range which can result in overwriting existing data in random locations). The number of supported OHs by any particular build is given in register GEM_AMC.GEM_SYSTEM.CONFIG.NUM_OF_OH, additionally the size of each RAM in units of 32bit words can be read with these registers:
GEM_AMC.CONFIG_BLASTER.STATUS.GBT_RAM_SIZE
GEM_AMC.CONFIG_BLASTER.STATUS.VFAT_RAM_SIZE
GEM_AMC.CONFIG_BLASTER.STATUS.OH_RAM_SIZE

Note that after loading the CTP7 firmware this RAM is empty, and therefore configuration streaming / "blasting" is disabled by default. Even though it doesn't really matter in this release, the software should write the configuration RAM with valid configuration first and then enable the blasting by writing 1 to GEM_AMC.CONFIG_BLASTER.CTRL.BLASTER_ENABLE

These registers are pointing to the base address of each RAM:
GEM_AMC.CONFIG_BLASTER.RAM.GBT
GEM_AMC.CONFIG_BLASTER.RAM.VFAT
GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA
Additionally there are convenience pointers to parts of each RAM corresponding to different OHs:
GEM_AMC.CONFIG_BLASTER.RAM.GBT_OH{0..N}
GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH{0..N}
GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH{0..N}

Below are descriptions of how the configuration data should be stored in each RAM.

GBT RAM
Each GBTX chip has 366 writable 8bit registers, but since word size of the RAM is 32bits, each RAM address stores 4 GBTX register values in LSB format, thus there are 92 RAM addresses dedicated for each GBTX chip. Since 366 is not divisible by 4, the last RAM address for any given GBTX will only be partially filled (top part of those addresses can be written with 0 or any other value).
e.g. lets say GBTX #0 of OH #1 has this configuration:
reg 0 = 0x12
reg 1 = 0x34
reg 2 = 0x56
reg 3 = 0x78
...
reg 360 = 0xaa
reg 361 = 0xbb
reg 362 = 0xcc
reg 363 = 0xdd
reg 364 = 0xee
reg 365 = 0xff

you should write that like this (pseudo-code):
write (GEM_AMC.CONFIG_BLASTER.RAM.GBT_OH1 + 0) 0x78563412
....
write (GEM_AMC.CONFIG_BLASTER.RAM.GBT_OH1 + 90) 0xddccbbaa
write (GEM_AMC.CONFIG_BLASTER.RAM.GBT_OH1 + 91) 0x0000ffee

Note that GEM_AMC.CONFIG_BLASTER.RAM.GBT_OH1 can be replaced with GEM_AMC.CONFIG_BLASTER.RAM.GBT + 1 * 92 * 3

VFAT RAM
The VFAT3 configuration RAM works in a very similar way to GBT RAM, but in VFAT3 each register value is 16bits wide, and there are 147 such registers per VFAT3, so each VFAT config RAM has 74 addresses dedicated for each VFAT3 where each address stores two VFAT3 register values in LSB format (also note that since 147 is not divisible by 2, the last RAM address of each VFAT will only be filled partially, and top part will be ignored).
e.g. lets say VFAT #10 of OH #2 has this configuration:
reg 0 = 0x1234
reg 1 = 0x5678
reg 2 = 0x9abc
reg 4 = 0xdeff
...
reg 144 = 0xaaaa
reg 145 = 0xbbbb
reg 146 = 0xcccc

you should write that like this (pseudo-code):
write (GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH2 + 10 * 74 + 0) 0x56781234
write (GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH2 + 10 * 74 + 1) 0xdeff9abc
....
write (GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH2 + 10 * 74 + 72) 0xbbbbaaaa
write (GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH2 + 10 * 74 + 73) 0x0000cccc

Note that GEM_AMC.CONFIG_BLASTER.RAM.VFAT_OH2 can be replaced with GEM_AMC.CONFIG_BLASTER.RAM.VFAT + 2 * 74 * 24

OH FPGA RAM
The OH FPGA configuration has to be streamed to random OH addresses, so it's necessary to store the OH register address as well as it's value (it's not necessary to store the address in GBT and VFAT RAM because the GBT and VFAT configuration spans a continuous address space). So for each OH register software should store a 32bit OH register address, and 32bit OH register value one after another in the OH FPGA config RAM. Note that the OH register addresses have to be local in OH firmware address space. Currently the OH FPGA config RAM is sized to accommodate 100 registers per OH, but this can be changed easily if required.

e.g. lets say we want to write these 2 registers of OH #3 to the OH FPGA configuration RAM:

address 0x0000000e (local in OH address space) corresponds to these registers in the address table:
FPGA.CONTROL.HDMI.SBIT_SEL0 (mask 0x0000001f): lets say we want value 0x1 here
FPGA.CONTROL.HDMI.SBIT_SEL1 (mask 0x000003e0): lets say we want value 0x2 here
FPGA.CONTROL.HDMI.SBIT_SEL2 (mask 0x00007c00): lets say we want value 0x3 here
FPGA.CONTROL.HDMI.SBIT_SEL3 (mask 0x000f8000): lets say we want value 0x4 here
FPGA.CONTROL.HDMI.SBIT_SEL4 (mask 0x01f00000): lets say we want value 0x5 here
FPGA.CONTROL.HDMI.SBIT_SEL5 (mask 0x3e000000): lets say we want value 0x6 here
after applying the mask to each of our desired values, the total 32bit value of this register would then be: 0x0c520c41

address 0x00002000 (local in OH address space) corresponds to this register in the address table:
FPGA.TRIG.CTRL.VFAT_MASK (mask 0x00ffffff): lets say we want value 0x00123456 here

you should write that like this (pseudo-code):
write (GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH3 + 0) 0x0000000e <-- reg address
write (GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH3 + 1) 0x0c520c41 <-- reg value of address 0xe
write (GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH3 + 2) 0x00002000 <-- reg address
write (GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH3 + 1) 0x00123456 <-- reg value of address 0x2000

GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA_OH3 can be replaced with GEM_AMC.CONFIG_BLASTER.RAM.OH_FPGA + 3 * 200

Most likely the configuration will be streamed to all OHs in parallel, but for any given OH the registers will be written in the same sequence as they appear in the config RAM, so if there's any register that has to be written before another register, just put it at lower address in the config RAM. There's probably not a good usecase for that, but who knows :)

Fiber mapping:
---=== CXP0 ===---
Fiber 1 / 13: OH0 GBT0
Fiber 2 / 14: OH0 GBT1
Fiber 3 / 15: OH0 GBT2
Fiber 4 / 16: OH1 GBT0
Fiber 5 / 17: OH1 GBT1
Fiber 6 / 18: OH1 GBT2
Fiber 7 / 19: OH2 GBT0
Fiber 8 / 20: OH2 GBT1
Fiber 9 / 21: OH2 GBT2
Fiber 10 / 22: OH3 GBT0
Fiber 11 / 23: OH3 GBT1
Fiber 12 / 24: OH3 GBT2

---=== CXP1 ===---
Fiber 1 / 13: OH4 GBT0
Fiber 2 / 14: OH4 GBT1
Fiber 3 / 15: OH4 GBT2
Fiber 4 / 16: OH5 GBT0
Fiber 5 / 17: OH5 GBT1
Fiber 6 / 18: OH5 GBT2
Fiber 7 / 19: OH6 GBT0
Fiber 8 / 20: OH6 GBT1
Fiber 9 / 21: OH6 GBT2
Fiber 10 / 22: OH7 GBT0
Fiber 11 / 23: OH7 GBT1
Fiber 12 / 24: OH7 GBT2

---=== CXP2 ===---
Fiber 1 / 13: OH8 GBT0
Fiber 2 / 14: OH8 GBT1
Fiber 3 / 15: OH8 GBT2
Fiber 4 / 16: OH9 GBT0
Fiber 5 / 17: OH9 GBT1
Fiber 6 / 18: OH9 GBT2
Fiber 7 / 19: OH10 GBT0
Fiber 8 / 20: OH10 GBT1
Fiber 9 / 21: OH10 GBT2
Fiber 10 / 22: OH11 GBT0
Fiber 11 / 23: OH11 GBT1
Fiber 12 / 24: OH11 GBT2

---=== MiniPOD MTP48 row 2 (fibers 25-36) ===---
Fiber 25: NO CONNECT
Fiber 26: NO CONNECT
Fiber 27: NO CONNECT
Fiber 28: NO CONNECT
Fiber 29: OH0 trigger link 0
Fiber 30: OH0 trigger link 1
Fiber 31: OH1 trigger link 0
Fiber 32: OH1 trigger link 1
Fiber 33: OH2 trigger link 0
Fiber 34: OH2 trigger link 1
Fiber 35: OH3 trigger link 0
Fiber 36: OH3 trigger link 1

---=== MiniPOD MTP48 row 1 (fibers 13-24) ===---
Fiber 13: OH4 trigger link 0
Fiber 14: OH4 trigger link 1
Fiber 15: OH5 trigger link 0
Fiber 16: OH5 trigger link 1
Fiber 17: OH6 trigger link 0
Fiber 18: OH6 trigger link 1
Fiber 19: OH7 trigger link 0
Fiber 20: OH7 trigger link 1
Fiber 21: OH8 trigger link 0
Fiber 22: OH8 trigger link 1
Fiber 23: OH9 trigger link 0
Fiber 24: OH9 trigger link 1

---=== MiniPOD MTP48 row 0 (fibers 1-12) ===---
Fiber 1: NO CONNECT
Fiber 2: NO CONNECT
Fiber 3: NO CONNECT
Fiber 4: NO CONNECT
Fiber 5: NO CONNECT
Fiber 6: NO CONNECT
Fiber 7: NO CONNECT
Fiber 8: NO CONNECT
Fiber 9: OH10 trigger link 0
Fiber 10: OH10 trigger link 1
Fiber 11: OH11 trigger link 0
Fiber 12: OH11 trigger link 1

---=== MiniPOD MTP48 row 3 (fibers 37-48) ===---
All of these are transmitters that go to EMTF

The GE1/1 MTP48-to-MTP24+MTP12 cable maps the MTP48 fibers to 24 OH trigger fibers on MTP24 in this order:
MTP24 fiber 13: OH0 trigger link 0
MTP24 fiber 14: OH0 trigger link 1
MTP24 fiber 15: OH1 trigger link 0
MTP24 fiber 16: OH1 trigger link 1
MTP24 fiber 17: OH2 trigger link 0
MTP24 fiber 18: OH2 trigger link 1
MTP24 fiber 19: OH3 trigger link 0
MTP24 fiber 20: OH3 trigger link 1
MTP24 fiber 21: OH4 trigger link 0
MTP24 fiber 22: OH4 trigger link 1
MTP24 fiber 23: OH5 trigger link 0
MTP24 fiber 24: OH5 trigger link 1
MTP24 fiber 1: OH6 trigger link 0
MTP24 fiber 2: OH6 trigger link 1
MTP24 fiber 3: OH7 trigger link 0
MTP24 fiber 4: OH7 trigger link 1
MTP24 fiber 5: OH8 trigger link 0
MTP24 fiber 6: OH8 trigger link 1
MTP24 fiber 7: OH9 trigger link 0
MTP24 fiber 8: OH9 trigger link 1
MTP24 fiber 9: OH10 trigger link 0
MTP24 fiber 10: OH10 trigger link 1
MTP24 fiber 11: OH11 trigger link 0
MTP24 fiber 12: OH11 trigger link 1

v1.16.0

-- 1.12.6 Set TXDLYBYPASS back to 0, and set TXPIPPMSEL to 1 only when shifting the PI phase and only if PA_GTH_SHIFT_USE_SEL is set to 1 (GTH reset doesn't complete if it's set to 1).
--        Also GTH PIPPM shift direction is configurable now. The delay between phase shifts is also configurable.
-- 1.12.7 Configurable TXDLYBYPASS, and a possibility to do manual shifts using registers
-- 1.12.8 Fixed a bug which previously prevented the GTH PIPPM shifting when doing a manual shift, so only the MMCM would be shifted
-- 1.12.9 Fix manual shift pulse length, and add manual shift support for GTH PIPPM only. Also set TXDLYBYPASS to 1 by default, because it really messes up the MMCM shifting
-- 1.12.10 Fix a bug in MMCM manual shifting, where a single write to manual shift enable register would sometimes not work or do multiple shifts
-- 1.12.11 Added a manual PLL reset control
-- 1.12.12 Added a way to do combined shifts: when PA_GTH_MANUAL_COMBINED, PA_GTH_MANUAL_OVERRIDE, and PA_MANUAL_OVERRIDE are set,
--         then pulsing the PA_GTH_MANUAL_SHIFT_EN will shift the GTH and also when necessary it will shift the MMCM in a way that keeps the internal GTH phase unchanged, but results in MMCM shift w.r.t. TTC backplane clock
--         (the MMCM shift direction is controlled automatically based on selected PA_GTH_MANUAL_SHIFT_DIR)
-- 1.12.13 Added flipflops for ipb_mosi inputs in the ipbus_slave.vhd to ease the timing on the ipbus path
-- 1.13.0  TTC commands are now latched using the backplane clock and buffered before decoding. The buffer is reset after a TTC module reset or a TTC resync command. This should avoid sampling bad TTC commands if the fabric clock phase shifts w.r.t. the backplane clock phase.
--         After a reset the buffer is filled up to a specified amount controlled by the BUF_DEPTH_AFTER_RESET register, which is set to 8 by default (total available buffer depth is 16).
--         Whenever the buffer depth exceeds a specified maximum depth (BUF_OOS_MAX_DEPTH, default = 9) or falls below a specified minimum depth level (BUF_OOS_MIN_DEPTH, default = 7), a TTS out-of-sync condition is triggered.
--         While the buffer is being filled initially after a reset, a TTS busy state is asserted (this should only last for BUF_DEPTH_AFTER_RESET clock cycles, which is 8 by default).
--         There are status registers to monitor the current buffer depth, buffer out of sync condition, and buffer busy condition.
--         Note that this introduces additional L1A latency, equal to BUF_DEPTH_AFTER_RESET + 1, so frontend latency should be adjusted accordingly.
-- 1.13.1  Wait until the TTC command FIFO has reset before enabling the writing
-- 1.13.2  Switched to distributed RAM FIFO for TTC command buffer
-- 1.13.3  Delay the check of OOS by 1 clock by starting to read 1 clock before asserting reset_done in the ttc command buffer logic
-- 1.13.4  Fixed a few bugs in DAQ: zero suppression was causing event size overflow; the chamber TTS state was not being propagated to top TTS (except for the last chamber); the TTS countdown period after reset was not working (not a problem really)
-- 1.14.0  Switched from end-of-event based on VFAT BC to OH EC and BC counters (default is using OH EC BC, but it can be switched back to VFAT BC with a registers). Also added OH EC BC counters to the readout in an unused spot of chamber trailer bits [31:0]
-- 1.14.1  In PA phase monitor, the sampling clock source was switched from the backplane clock to the jitter cleaned MMCM clock. This is to check if the spread on the phase measurement would improve.
-- 1.15.0  DAQ input processor is now also counting the number of zero suppressed VFAT words and putting that into the datastream, using a spot previously dedicated to per chamber zero suppression flags (bits [51:40] in chamber header). DAQ format version has been changed from 0 to 1 to reflect that.
--         Also registers are available to read the current, min, and max "VFAT live word count", which is the number of non-zero-suppressed + zero-suppressed 64bit words (total should be equal to 24 * 3 in normal conditions regardless of zero suppression setting)
--         Also this version introduces autokilling of bad links (is enabled by default, but can be disabled). When a given link times out more than a certain number of times in a row (this parameter is configurable, and is set to 100 by default), then this input will be autokilled.
--         The autokill mask is reset during each resync. The autokill mask, and the individual link timeout counters (both consecutive and total) can be read with registers.
--         Default values of the TTC command buffer min and max for OOS have been set to 2 and 4 respectively (previously these were set at 3 and 3).
-- 1.15.1  Minor fixes related to input autokill: when an input is autokilled, also mask its TTS state, and also the consecutive timeout count for the first input had a bug related to resetting which was fixed
-- 1.15.2  TTC buffer monitoring counters added: number of OOS instances, number of underflow and overflow instances, number of seconds since last OOS, last and max duration of the OOS state in clock cycles
-- 1.15.3  ILA added to check the clocks and fifo signals in the TTC command buffer
-- 1.16.0  Updated the Chip2Chip interface to be compatible with the new linux versions that support gemloader 

v3.5.1

Added flipflops for ipb_mosi inputs in the ipbus_slave.vhd to ease the timing on the ipbus path.
This release also includes a 4 OH version

v1.12.5

-- 1.12.3 GTH PI clock phase adjustment implemented when shifting the MMCM outputs (which affects the TXUSRCLK phase). This should keep the GTH PI and TXUSRCLK reasonably in phase
-- 1.12.4 Added GTH shift count and GTH shift error registers. Also set TXDLYBYPASS = 1, and TXPIPPMSEL = 1
-- 1.12.5 Inverted the GTH shift direction w.r.t. MMCM shift direction, because in MMCM we're shifting the feedback clock, which actually shifts the outputs in the opposite direction..