The code of the alignment bias is not consistent with Equation 9.

[FacePoluke]

The values outside of the diagonal are 1, not -inf.

[ChengyuanYan]

the function returns a boolean mask so the placeholder value doesn't really matter. When the transformer computes attention, places that have non-zero value (i.e. True in the returned matrix) will be masked out, which is somewhat equivalent to adding -inf to the attention value