KG's Blog – Welcome

Tuning LLVM's SLP Vectorizer Cost Model

Sun, 24 May 2026 11:00:55 -0400

Similar to my last post, this writeup covers how I solved a performance regression on LLVM by analyzing a benchmark from a RISCV target.

TLDR
A recent LLVM patch introduced ordered vector reductions to replace a chain of scalar fadds, but it triggered a performance regression on a benchmark by failing to account for cost of building the initial vector per iteration. This in turned caused unprofitable code to be deemed “profitable.”
PR, Issue

The Regression

Looking at Igalia’s LNT instance for the BPI-F3, I noticed this particular benchmark with a delta of 89%. Specifically, there was an increase in ~26% issued instructions and a ~48% increase in cycles.

I have attached two more pictures right below, with the first one being the assembly of a basic block from the older build and the corresponding assembly from the newer build.

Info

Bn here refers to Billions of cycles. This basic block is basically taking twice as many cycles to execute.

We can see that that newer build of LLVM is performing a sequence of fsd instructions, also known as Float Store Double. It’s essentially storing the floating point values from those registers onto the stack. Specifically, it’s storing the value at the address s1 + 0x80.

From a preceding basic block that I have not included here, I know that value of the register a5 to be equal to s1 + 0x80 from this instruction.

addi	a5, s1, 0x80

The Vector Load Instruction vle64.v is loading the values from memory at the address at a5 (s1 + 0x80) into the vector register v16.

v16 = M[a5]

Finally, it executes the vfredosum.vs instruction (Ordered floating-point sum), which performas the following for a vector register of size VL.

\[ vd[0] = \left( \dots \left( \left( vs1[0] + vs2[0] \right) + vs2[1] \right) + \dots + vs2[VL-1] \right) \]

The new codegen is basically trying to replace the ordered fadd instructions in the first basic block with this vector sum reduction instruction. I hope this diagram may illustrate this better, with what was previously happening versus what is currently occurring. From the images above, it can be observed that the new code is significantly more expensive in terms of cycles.

    graph LR
    %% Left-to-Right top level allows parallel tracks to scale height independently
    V_IN["Original Source Data<br/><i>(Residing in Scalar Registers)</i>"]

    %% Link to Scalar Track
    V_IN -->|Old Execution| S1
    
    %% Link to Vector Track
    V_IN -->|New Execution| FSD1

    %% Left Side Track: Original Scalar Chain
    subgraph ScalarChain ["Original Intent: Ordered fadd Chain"]
        direction TB
        S1["fadd (START_VAL + SCALAR_VAL_0)"] --> S2["fadd (Result + SCALAR_VAL_1)"]
        S2 --> S3["fadd (Result + SCALAR_VAL_2)"]
        S3 --> S4["fadd (Result + SCALAR_VAL_3)<br/><code>FINAL_SCALAR_SUM</code>"]
    end

    %% Right Side Track Part 1: Memory Gather (Now cleanly scales to its actual contents)
    subgraph Gather ["Memory Gather (Stack Spilling Penalty)"]
        direction TB
        FSD1["Store SCALAR_VAL_0 to s1 + 0x80"]
        FSD2["Store SCALAR_VAL_1 to s1 + 0x88"]
        FSD3["Store SCALAR_VAL_2 to s1 + 0x90"]
        FSD4["Store SCALAR_VAL_3 to s1 + 0x98"]
    end

    subgraph VectorOps ["Vector Load & Reduction"]
        direction TB
        VLE["Vector Load<br/><i>(Loads from memory into vector register v16)</i>"] --> VFRED["Vector Ordered Sum Reduction<br/><i>(vfredosum)</i>"]
    end

    %% Connect the two vector halves sequentially
    FSD4 -->VLE

    %% --- STYLING BLOCK ---
    style ScalarChain fill:none,stroke:#888888,stroke-dasharray: 5 5
    
    %% Gather Penalty (Soft Red Glow)
    style Gather fill:#b71c1c18,stroke:#ef4444,stroke-width:2px,rx:8,ry:8
    
    %% Vector Operations (Soft Blue Glow)
    style VectorOps fill:#0d47a118,stroke:#3b82f6,stroke-width:2px,rx:8,ry:8

Info

If you visit the link yourself, you may notice that there is also another basic block further down that also has a significant increase in cycles compared to its older counterpart. I chose not to include that as both of them are identical, so fixing one fixes the other.

The Where?

To narrow down where these new fsd and vfredosum.vs instructions are introduced, I ran the command below to get emit the LLVM IR. The output of this commend will give us the intermediate representation produced by the middle-end. If we can observe IR code that would result in those instructions, we can rule out the backend.

$lbd/bin/clang -O3 \
  --target=riscv64-unknown-linux-gnu \
  -march=rva22u64_v \
  --gcc-toolchain=/usr \
  --sysroot=/usr/riscv64-linux-gnu \
  -I. \
  -DFP_ABSTOLERANCE=1e-5 \
  -S -emit-llvm seidel-2d.c \
  -o seidel-2d.ll

for.body8.i:                                      ; preds = %for.body8.i, %for.cond5.preheader.i
  %18 = phi double [ %.pre117.i, %for.cond5.preheader.i ], [ %26, %for.body8.i ]
  %19 = phi double [ %.pre116.i, %for.cond5.preheader.i ], [ %18, %for.body8.i ]
  %20 = phi double [ %.pre115.i, %for.cond5.preheader.i ], [ %25, %for.body8.i ]
  %21 = phi double [ %.pre114.i, %for.cond5.preheader.i ], [ %div.i40, %for.body8.i ]
  %22 = phi double [ %.pre113.i, %for.cond5.preheader.i ], [ %24, %for.body8.i ]
  %23 = phi double [ %.pre.i, %for.cond5.preheader.i ], [ %22, %for.body8.i ]
  %indvars.iv.i38 = phi i64 [ 1, %for.cond5.preheader.i ], [ %indvars.iv.next.i39, %for.body8.i ]
  %add.i = fadd double %22, %23
  %indvars.iv.next.i39 = add nuw nsw i64 %indvars.iv.i38, 1
  %arrayidx23.i = getelementptr inbounds nuw [8 x i8], ptr %arrayidx.i37, i64 %indvars.iv.next.i39
  %24 = load double, ptr %arrayidx23.i, align 8, !tbaa !14
  %arrayidx34.i = getelementptr inbounds nuw [8 x i8], ptr %17, i64 %indvars.iv.i38
  %arrayidx40.i = getelementptr inbounds nuw [8 x i8], ptr %17, i64 %indvars.iv.next.i39
  %25 = load double, ptr %arrayidx40.i, align 8, !tbaa !14
  %arrayidx60.i = getelementptr inbounds nuw [8 x i8], ptr %arrayidx44.i, i64 %indvars.iv.next.i39
  %26 = load double, ptr %arrayidx60.i, align 8, !tbaa !14
  %27 = insertelement <8 x double> poison, double %24, i32 0
  %28 = insertelement <8 x double> %27, double %add.i, i32 1
  %29 = insertelement <8 x double> %28, double %21, i32 2
  %30 = insertelement <8 x double> %29, double %20, i32 3
  %31 = insertelement <8 x double> %30, double %25, i32 4
  %32 = insertelement <8 x double> %31, double %19, i32 5
  %33 = insertelement <8 x double> %32, double %18, i32 6
  %34 = insertelement <8 x double> %33, double %26, i32 7
  %35 = call double @llvm.vector.reduce.fadd.v8f64(double -0.000000e+00, <8 x double> %34)
  %div.i40 = fdiv double %35, 9.000000e+00
  store double %div.i40, ptr %arrayidx34.i, align 8, !tbaa !14
  %exitcond.not.i41 = icmp eq i64 %indvars.iv.next.i39, 1999
  br i1 %exitcond.not.i41, label %for.inc66.i, label %for.body8.i, !llvm.loop !25

We can observe here a chain of insertelement instructions building an <8 x double> vector. The instruction follows this format:

%result = insertelement <vector_type> <source_vector>, <scalar_value>, <index>

Using the snippet above, we can see that %27 is the initial vector. Because there is no prior value, the source is marked as poison, the scalar value it’s inserting is the value %24, and the index is 0. From there, each subsequent instruction feeds the result of the previous line its source vector operand.

Once the vector is completely packed at %34, it serves as an operand to the vector.reduce.fadd intrinsic (maps to vfredosum). The LLVM LangRef explains how this intrinsic sums up the elements in the second operand (34%), and adds the first operand (0.0), and stores the result (%35). The pseudocode provided by the LangRef probably explains this better.

float sequential_fadd(start_value, input_vector)
  result = start_value // This is -0.00000
  for i = 0 to length(input_vector) // Traverses every element in %34
    result = result + input_vector[i]
  return result

Intrinsic

LLVM intrinsics are built-in functions that map directly to low-level architecture instructions, but can be hard to express in normal LLVM IR semantics. They give the compiler a better understanding of a hardware operation, allowing it to generate more efficient assembly.

Since the behaviour is already introduced by the middle-end, we can rule out the RISCV backend. I can confidently say that it was some middle-end pass causing this behaviour.

This gives us a better idea of what to isolate. I ran git log on the dates of when the builds were ran, and grepped for keywords. After hitting a couple of dead-ends, I found a promising lead when I filtered specifically for reduction.

git log --since="2026-05-19 00:00:00 +0000" --until="2026-05-20 00:13:39 +0000" --oneline | grep reduc

e28e7ec30357 Reland [VectorCombine] foldShuffleChainsToReduce - add support for partial vector reductions (#197659)
dfa05b675eee [VectorCombine] Fold reduce.add == 0 into reduce.[or,umax] == 0 (#180001)
230980947083 [SLP] Support ordered fadd reduction via reduction intrinsics
2d9407bb56b7 [X86] Add handling for sub-128bit minmax reductions (#198319)

The third commit immediately stood out, as its description would perfectly explain the new codegen. After building LLVM locally before and with this commit, I was able to determine that the commit was the culprit.

The Why?

For some reason, the SLP Vectorizer thinks it more profitable to performed the fadd reduction here when we know from the benchmark that its not. The key things that author added in that commit were the following:

matchOrderedReduction
- Responsible for finding possible instructions for ordered reduction.
tryToReduceOrdered
- This is the function that actually checks if its profitable, and commits to the ordered reduction if it is.

SLP Vectorizer

I think the LLVM docs does a decent job at a high-level overview of what the SLPVectorizer does. LLVM has two vectorizers, and the SLP Vectorizer is one of them. The pass’ main goal is to combine independent instructions into vector instructions. This is a pretty important in terms of performance. By vectorizing what was previously scalar instructions, you can perform the same operations on multiple values at once. This is especially beneficial when operations within a loop can be vectorized, as these tend to be the hotspots within a program.

In this case though, the SLP Vectorizer was vectorizing a sequence of ordered instructions. I think the commit message itself will elucidate the goal of this commit better.

Previously, the SLP vectorizer could only vectorize ordered reductions by keeping the original scalar chain and emitting extractelement instructions. The new path replaces the scalar chain with a vector ordered reduction intrinsic (where profitable), which allows the backend to lower it more efficiently.

The following test case from the PR further illustrates the goal of the commit.

- ; CHECK-NEXT:    [[TMP28:%.*]] = extractelement <4 x float> [[TMP17]], i32 0
- ; CHECK-NEXT:    [[TMP32:%.*]] = extractelement <4 x float> [[TMP17]], i32 1
- ; CHECK-NEXT:    [[ADD30:%.*]] = fadd float [[TMP28]], [[TMP32]]
- ; CHECK-NEXT:    [[TMP20:%.*]] = extractelement <4 x float> [[TMP17]], i32 2
- ; CHECK-NEXT:    [[ADD31:%.*]] = fadd float [[ADD30]], [[TMP20]]
- ; CHECK-NEXT:    [[TMP30:%.*]] = extractelement <4 x float> [[TMP17]], i32 3
- ; CHECK-NEXT:    [[ADD33:%.*]] = fadd float [[ADD31]], [[TMP30]]
+ ; CHECK-NEXT:    [[ADD33:%.*]] = call float @llvm.vector.reduce.fadd.v4f32(float -0.000000e+00, <4 x float> [[TMP17]])

Before this commit, LLVM had to extract elements out of a vector to perform a chain of scalar fadd. This commit notices that it’s cheaper to keep them in the vector, and use the reduction intrinsic (which maps to vfredosum on RISCV).

    graph TD
    %% Top Level Input
    V_IN["Source Vector Input<br/><code>VECTOR_INPUT</code>"]

    %% Split Path
    V_IN -->|Old Code Generation| OldBehavior
    V_IN -->|Optimized SLP Rewrite| NewBehavior

    %% Left Side Path: Old Behavior (The Costly Extraction Chain)
    subgraph OldBehavior ["Before: Extraction Penalty"]
        direction TB
        
        %% Extraction Row
        subgraph Extractions ["Explicit Extractions"]
            E0["extractelement (EL0)"]
            E1["extractelement (EL1)"]
            E2["extractelement (EL2)"]
            E3["extractelement (EL3)"]
        end
        
        %% Chain
        A0["fadd (EL0 + EL1)"]
        A1["fadd (Result + EL2)"]
        A2["fadd (Result + EL3)<br/><code>FINAL_SCALAR_SUM</code>"]
        
        %% Wire it up
        E0 & E1 --> A0
        A0 & E2 --> A1
        A1 & E3 --> A2
    end

    %% Right Side Path: New Behavior (The Native Intrinsic)
    subgraph NewBehavior ["Direct Reduction"]
        direction TB
        INTRINSIC["<b>@llvm.vector.reduce.fadd</b><br/><i>Executes natively inside the vector unit</i><br/><code>FINAL_SCALAR_SUM</code>"]
    end

    %% --- STYLING BLOCK ---
    style OldBehavior fill:none,stroke:#888888,stroke-dasharray: 5 5
    style NewBehavior fill:none,stroke:#888888,stroke-dasharray: 5 5
    style Extractions fill:none,stroke:none
    
    %% Highlight the bad overhead path in red
    style E0 fill:#b71c1c18,stroke:#ef4444,stroke-width:1px
    style E1 fill:#b71c1c18,stroke:#ef4444,stroke-width:1px
    style E2 fill:#b71c1c18,stroke:#ef4444,stroke-width:1px
    style E3 fill:#b71c1c18,stroke:#ef4444,stroke-width:1px
    
    %% Highlight the clean vector intrinsic win in green
    style INTRINSIC fill:#1b5e2018,stroke:#4caf50,stroke-width:2px,rx:8,ry:8

TreeEntry

The TreeEntry is a datastructure used to hold relevant data to determine which scalar operations can be combined into vector operations. The log below was printed using TE.dump() on the benchmark.

==== [SLP Cost Loop] Examining TreeEntry (Idx: 0) ====
0.
Scalars: 
    %45 = load double, ptr %arrayidx23.i75, align 8, !tbaa !16
    %add.i73 = fadd double %43, %44
    %42 = phi double [ %.pre114.i65, %for.cond5.preheader.i58 ], [ %div.i86, %for.body8.i71 ]
    %41 = phi double [ %.pre115.i67, %for.cond5.preheader.i58 ], [ %46, %for.body8.i71 ]
    %46 = load double, ptr %arrayidx40.i80, align 8, !tbaa !16
    %40 = phi double [ %.pre116.i68, %for.cond5.preheader.i58 ], [ %39, %for.body8.i71 ]
    %39 = phi double [ %.pre117.i70, %for.cond5.preheader.i58 ], [ %47, %for.body8.i71 ]
    %47 = load double, ptr %arrayidx60.i84, align 8, !tbaa !16
State: NeedToGather
MainOp: NULL
AltOp: NULL
VectorizedValue: NULL
ReuseShuffleIndices: Empty
ReorderIndices: 
UserTreeIndex: <invalid>

I’m not a 100% sure why there are phi instructions within Scalars, but I’m fairly sure it’s because the function matchOrderedReduction, which is responsible for finding ordered operations, walks back to the root of the instructions. The comments on this function may explain this better.

  /// Analyze whether \p Root forms a linearized ordered reduction chain.
  /// If \p MatchLHS is true, analyzes LHS-associated (left-linearized) chains
  /// where the chain recurses on the LHS and the RHS at each level is a leaf:
  ///   ((((v0 op v1) op v2) op v3) op v4)
  /// If \p MatchLHS is false, analyzes RHS-associated (right-linearized) chains
  /// where the chain recurses on the RHS and the LHS at each level is a leaf:
  ///   (v0 op (v1 op (v2 op (v3 op v4))))
  /// Leaf values are stored in ReducedVals.back() in accumulation order
  /// (innermost pair first, outermost last), e.g. [v0,v1,v2,v3,v4] for
  /// LHS-associated and [v4,v3,v2,v1,v0] for RHS-associated chains.
  /// \returns true if the chain is a valid ordered reduction.

The key thing to notice is that the UserTreeIndex is invalid. tryToReduceOrdered builds an isolated tree by calling V.buildTree(VL, IgnoreList) specifically for the ordered reduction candidates. Since this is the root, there is no parent, so EI (EdgeInfo) would be null.

Tracing the logs to the code

I first raised an issue and pinged the author of the commit with a kind of hacky solution. He suggested an inaccurate TTI (Target Transform Info) for this particular target.

Issue: https://github.com/llvm/llvm-project/issues/199267

Target Transform Info (TTI)

LLVM middle-end tries to be target-agnostic. But it’s easy to imagine scenarios where we would need info on the target during the middle-end. For example, building the vector in this benchmark might be cheaper on other targets. Target Transform Info provides hooks that can be overridden by backends to provide info to the middle-end.

With this in mind, I kept investigating the code alongside the logs. I noticed this piece of code from the diff.

+      LLVM_DEBUG(dbgs() << "SLP: Vectorizing ordered reduction at cost:" << Cost
+                        << ". (HorRdx)\n");

When the SLPVectorizer commits to performing an ordered reduction, it prints that debug statement alongside the cost. Searching through the log for this debug message, I found two instances of “Vectorizing ordered…”, which corresponds to the two basic blocks in this benchmark.

SLP: Adding cost 15 for bundle Idx: 0, n=8 [  %24 = load double, ptr %arrayidx23.i, align 8, !tbaa !16, ..].
SLP: Current total cost = 15
SLP: Adding cost -4 for reduction of n=8 [  %24 = load double, ptr %arrayidx23.i, align 8, !tbaa !16, ..] (It is a splitting reduction)
Scale 1996002000 For entry 0
SLP: Spill Cost = 0.
SLP: Extract Cost = 0.
SLP:  Reduction Cost = -7984008000.
SLP: Total Cost = -7984007985.
SLP: Found cost = -7984007985 for ordered reduction
SLP: Vectorizing ordered reduction at cost:-7984007985. (HorRdx)

From these logs, we can observe a couple of things:

The value of Scale, which is the number of iterations in a loop, is 1996002000.
The Reduction Cost was -7984008000, which I deduce is from the -4 being multiplied by the Scale value
The TreeCost which encompasses the cost of the bundle in this case, which corresponds to the instructions building the vector to reduce it, is 15
The total cost was -7894007895, the TreeCost subtracted by the ReductionCost

\[ \begin{aligned} &\text{TreeCost} = 15 \\ &\text{ReductionCost} = -4 \\ &\text{Scale} = 1,996,002,000 \\[1em] &\text{ReductionCost}' = \text{ReductionCost} \times \text{Scale} \\ &\text{Total Cost} = \text{TreeCost} + \text{ReductionCost}' \\ &\text{Total Cost} = 15 + (-7,984,008,000) \\ &\text{Total Cost} = -7,984,007,985 \end{aligned} \]

I hope the problem is clearer now. In this benchmark, the TreeCost - the cost to build the vector - is not scaled alongside the ReductionCost. This results in really big negative value, leading the SLPVectorizer to think this ordered reduction is profitable.

If you’re wondering where exactly the Scale came to be, this snippet from the source code can explain this.

// seidel-2d.h
/* Default to LARGE_DATASET. */
# if !defined(MINI_DATASET) && ...
#  define LARGE_DATASET
# endif

#  ifdef LARGE_DATASET
#   define TSTEPS 500
#   define N 2000
#  endif

// seidel-2d.c
for (t = 0; t <= _PB_TSTEPS - 1; t++)
  for (i = 1; i <= _PB_N - 2; i++)
    for (j = 1; j <= _PB_N - 2; j++)

\[ \begin{aligned} &\text{TSTEPS} = 500 \\ &\text{_PB_N} = 2000 \\[1em] &\text{Total Iterations} = \text{TSTEPS} \times (\text{_PB_N} - 2) \times (\text{_PB_N} - 2) \\ &\text{Total Iterations} = 500 \times 1998 \times 1998 \\ &\text{Total Iterations} = 1,996,002,000 \end{aligned} \]

If we look at the function getTreeCost, we can see how ReductionCost is scaled.

InstructionCost BoUpSLP::getTreeCost(InstructionCost TreeCost,
                                     ArrayRef<Value *> VectorizedVals,
                                     InstructionCost ReductionCost,
                                     Instruction *RdxRoot) {
  
  ...

  InstructionCost Cost = TreeCost;

  SmallDenseMap<std::tuple<const TreeEntry *, Value *, Instruction *>, uint64_t>
      EntryToScale;
  auto ScaleCost = [&](InstructionCost C, const TreeEntry &TE,
                       Value *Scalar = nullptr, Instruction *U = nullptr) {
    if (!C.isValid() || C == 0)
      return C;
    uint64_t &Scale =
        EntryToScale.try_emplace(std::make_tuple(&TE, Scalar, U), 0)
            .first->getSecond();
    if (!Scale)
      Scale = getScaleToLoopIterations(TE, Scalar, U);
    LLVM_DEBUG(dbgs() << "Scale " << Scale << " For entry " << TE.Idx << "\n");
    return C * Scale;
  };
  Instruction *ReductionRoot = RdxRoot;
  if (UserIgnoreList) {
    // Scale reduction cost to the factor of the loop nest trip count.
    ReductionCost = ScaleCost(ReductionCost, *VectorizableTree.front().get(),
                              /*Scalar=*/nullptr, ReductionRoot);
  }

  // Add the cost for reduction.
  Cost += ReductionCost;

  ...

}

ReductionCost is being multiplied by the valued returned by getScaleToLoopIterations. The function getScaleToLoopIterations also has access to *RdxRoot, which gives getScaleToLoopIterations awareness of the parent block, and lets it figure out the number of iterations if the block is a loop.

To figure out why the TreeCost is not being scaled, lets again compare the logs to the code. By searching for “for bundle:”, I was able to find this snippet of the code in a function called calculateTreeCostAndTrimNonProfitable. As the name suggests, this is the function responsible for calculating the cost of TreeCost.

InstructionCost
BoUpSLP::calculateTreeCostAndTrimNonProfitable(ArrayRef<Value *> VectorizedVals,
                                               Instruction *RdxRoot) {

    ...

    InstructionCost C = getEntryCost(&TE, VectorizedVals, CheckedExtracts);
    uint64_t Scale = 0;
    bool CostIsFree = C == 0;

    ...

    if (!CostIsFree && !Scale) {
      Scale = IsGatherLike ? getGatherNodeEffectiveScale(TE)
                           : getScaleToLoopIterations(TE);
      C *= Scale;
      EntryToScale.try_emplace(&TE, Scale);
      if (!TE.isGather() && TE.hasState()) {
        PrevVecParent = TE.getMainOp()->getParent();
        PrevScale = Scale;
      }
    }
    Cost += C;
    NodesCosts.try_emplace(&TE, C);
    LLVM_DEBUG(dbgs() << "SLP: Adding cost " << C << " for bundle "
                      << shortBundleName(TE.Scalars, TE.Idx) << ".\n"
                      << "SLP: Current total cost = " << Cost << "\n");

    ...

}

To get a better idea of what was happening, I added debug logs. From these logs, I determined that getGatherNodeEffectiveScale is called to calculate the Scale value here. This function then calls getScaleToLoopIterations.

uint64_t BoUpSLP::getScaleToLoopIterations(const TreeEntry &TE, Value *Scalar,
                                           Instruction *U) {
  BasicBlock *Parent = nullptr;
  
  ... 

  } else if (TE.isGather() || TE.State == TreeEntry::SplitVectorize) {
    EdgeInfo EI = TE.UserTreeIndex;
    while (EI.UserTE) {
      ...
    }
    if (!Parent)
      return 1;
  } else {

  ...

}

In this case, EI.UserTE is null, so the (!Parent) would return true and the function would just return 1.

Landing a fix

I’ll be upfront in that the author, alexey-bataev, heavily guided me towards this fix. Take a look at the issue if you want to see my initial ideas on trying to solve this.

Here is a link to the final PR as well as the precommit test:

The solution was to pass the RdxRoot instruction to calculateTreeCostAndTrimNonProfitable, and subsequently to getGatherNodeEffectiveScale and getScaleToLoopIterations, so that they have the loop context needed to correctly scale TreeCost.

In simpler terms, Scale was previously being assigned the value of 1 because the pass did not have access to the parent block of the insertelement instructions. This patch addresses that.

    graph TD
    %% Main execution flow container
    subgraph Pipeline ["The Fix: Passing RdxRoot"]
        direction TB
        
        %% Step 1
        A["<b>tryToReduceOrdered</b><br/>Identifies reduction and holds reference to <code>RdxRootInst</code>"]
        
        %% Step 2
        B["<b>calculateTreeCostAndTrimNonProfitable</b><br/>Accepts new parameter: <code>Instruction *RdxRoot</code>"]
        
        %% Step 3 (The Ternary Switch)
        C{"Is the node<br/>Gather-Like?"}
        
        %% Step 4a
        D["<b>getGatherNodeEffectiveScale(TE, RdxRoot)</b><br/><i>(Applies to Root Node at TE.Idx == 0)</i>"]
        
        %% Step 4b
        E["<b>getScaleToLoopIterations(TE)</b>"]
        
        %% Step 5
        F["<b>getScaleToLoopIterations(TE, nullptr, U)</b><br/>Bypasses missing tree parent by anchoring to <code>U</code> (RdxRoot)"]

        %% Define flow connections
        A -->|Passes RdxRootInst| B
        B --> C
        C -->|Yes: Node is isolated| D
        C -->|No: Standard node| E
        D -->|Passes U to find loop context| F
    end

    %% --- STYLING BLOCK ---
    %% Neutral structural wrapper for the pipeline
    style Pipeline fill:none,stroke:#888888,stroke-dasharray: 5 5
    
    %% Standard nodes styling
    style A fill:none,stroke:#888888,stroke-width:1px
    style B fill:none,stroke:#888888,stroke-width:1px
    style C fill:none,stroke:#888888,stroke-width:1px
    style E fill:none,stroke:#888888,stroke-width:1px
    
    %% Highlights for where the fix actively changes behavior (Soft Green Glow)
    style D fill:#1b5e2018,stroke:#4caf50,stroke-width:2px,rx:8,ry:8
    style F fill:#1b5e2018,stroke:#4caf50,stroke-width:2px,rx:8,ry:8

diff --git a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
index 54a4d6b68b2e..d90eba7118b3 100644
--- a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
+++ b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
@@ -2176,7 +2176,8 @@ public:
   /// Calculates the cost of the subtrees, trims non-profitable ones and returns
   /// final cost.
   InstructionCost
-  calculateTreeCostAndTrimNonProfitable(ArrayRef<Value *> VectorizedVals = {});
+  calculateTreeCostAndTrimNonProfitable(ArrayRef<Value *> VectorizedVals = {},
+                                        Instruction *RdxRoot = nullptr);
 
   /// \returns the vectorization cost of the subtree that starts at \p VL.
   /// A negative number means that this is profitable.
@@ -3996,7 +3997,8 @@ private:
   /// LICM hoisting that optimizeGatherSequence() performs after vectorization
   /// for inserts with loop-invariant operands. Falls back to the whole-entry
   /// scale when per-lane information is unavailable or the feature is off.
-  uint64_t getGatherNodeEffectiveScale(const TreeEntry &TE);
+  uint64_t getGatherNodeEffectiveScale(const TreeEntry &TE,
+                                       Instruction *U = nullptr);
 
   /// Get the loop nest for the given loop \p L.
   ArrayRef<const Loop *> getLoopNest(const Loop *L);
@@ -16455,14 +16457,15 @@ uint64_t BoUpSLP::getLoopNestScale(const Loop *L) {
   return std::max<uint64_t>(1, Scale);
 }
 
-uint64_t BoUpSLP::getGatherNodeEffectiveScale(const TreeEntry &TE) {
+uint64_t BoUpSLP::getGatherNodeEffectiveScale(const TreeEntry &TE,
+                                              Instruction *U) {
   // Only meaningful for gather/buildvector-like entries; the per-lane
   // insertelements that make up such an entry are LICM-hoistable by
   // optimizeGatherSequence() when their operand is loop-invariant.
   assert((TE.isGather() || TE.State == TreeEntry::SplitVectorize) &&
          "Expected gather/split tree entry.");
 
-  uint64_t BaseScale = getScaleToLoopIterations(TE);
+  uint64_t BaseScale = getScaleToLoopIterations(TE, nullptr, U);
   if (!PerLaneGatherScale || LoopAwareTripCount == 0 || BaseScale <= 1)
     return BaseScale;
 
@@ -16487,7 +16490,8 @@ uint64_t BoUpSLP::getGatherNodeEffectiveScale(const TreeEntry &TE) {
     if (isConstant(V))
       continue;
     ++N;
-    uint64_t LaneScale = std::min(getScaleToLoopIterations(TE, V), BaseScale);
+    uint64_t LaneScale =
+        std::min(getScaleToLoopIterations(TE, V, U), BaseScale);
     Sum = SaturatingAdd(Sum, LaneScale, &Overflow);
     if (Overflow)
       return BaseScale;
@@ -18839,8 +18843,9 @@ static T *performExtractsShuffleAction(
   return Prev;
 }
 
-InstructionCost BoUpSLP::calculateTreeCostAndTrimNonProfitable(
-    ArrayRef<Value *> VectorizedVals) {
+InstructionCost
+BoUpSLP::calculateTreeCostAndTrimNonProfitable(ArrayRef<Value *> VectorizedVals,
+                                               Instruction *RdxRoot) {
   // FIXME: support buildvector of the gather nodes with struct types.
   if (any_of(VectorizableTree, [&](const std::unique_ptr<TreeEntry> &TE) {
         return TE->isGather() &&
@@ -18960,8 +18965,10 @@ InstructionCost BoUpSLP::calculateTreeCostAndTrimNonProfitable(
       }
     }
     if (!CostIsFree && !Scale) {
-      Scale = IsGatherLike ? getGatherNodeEffectiveScale(TE)
-                           : getScaleToLoopIterations(TE);
+      Scale =
+          IsGatherLike
+              ? getGatherNodeEffectiveScale(TE, TE.Idx == 0 ? RdxRoot : nullptr)
+              : getScaleToLoopIterations(TE);
       C *= Scale;
       EntryToScale.try_emplace(&TE, Scale);
       if (!TE.isGather() && TE.hasState()) {
@@ -29837,7 +29844,8 @@ public:
 
       V.transformNodes();
       V.computeMinimumValueSizes();
-      InstructionCost TreeCost = V.calculateTreeCostAndTrimNonProfitable(VL);
+      InstructionCost TreeCost =
+          V.calculateTreeCostAndTrimNonProfitable(VL, RdxRootInst);
       V.buildExternalUses(LocalExternallyUsedValues);
 
       InstructionCost ReductionCost =

Performance

The image below captures the post-patch performance improvement. Although, if we compare this to the original baseline I displayed at the beginning of this writeup, we can see an increase in issued instructions, and a slight decrease in vector instructions.

However, the total cycles stays about the same to the baseline! 🥹

Tracking down a 25% Regression on LLVM RISC-V

Thu, 09 Apr 2026 15:00:00 -0400

Similar to the previous post, this post covers my analysis of a benchmark on RISC-V targets. Unlike the previous post, I was able to land a patch to eliminate the performance gap to GCC (for this benchmark)!

TLDR

A recent LLVM commit improved isKnownExactCastIntToFP to fold fpext(sitofp x to float) to double into a direct uitofp x to double cast, but this inadvertently broke a downstream narrowing optimization in visitFPTrunc that relied on the fpext to narrow a double to float, causing a ~24% performance regression on RISC-V targets, where fdiv.d (33 cycle latency) was emitted instead of fdiv.s (19 cycle latency).

My fix extends getMinimumFPType with range analysis to recognize that fptrunc(uitofp x double) to float can be reduced to uitofp x to float, restoring the narrowing optimization.

Analysis

I was looking at Igalia’s site comparing the performance of LLVM to GCC on RISCV targets, and I noticed this particular benchmark.

As shown in the image below, LLVM requires about ~8% more cycles than GCC for that specific benchmark on the SiFive P550 CPU.

I have included snippets of the relevant basic block assembly. Practically all the cycles were spent on the assembly below.

LLVM

GCC

From the two assembly, it wasn’t immediately obvious to me why GCC was doing better. They seemed almost identical, and if anything, LLVM was able to optimize the branch logic here. The big difference I did notice was that LLVM was emitting an fdiv.d, a division with double precision floating point or an f64.

This did seem promising so I decided to run llvm-mca on the source code to get a better idea of what is happening. Note that I am using llvm-mca built from source that was a couple of days old from upstream. From the analysis done by llvm-mca, I noticed that the fdiv.d instruction made no appearance in the loop. Although I did not show it above, both GCC and LLVM contained an fdiv.d instruction in a later basic block but, this was outside the main loop and therefore not relevant to the performance difference.

Info

llvm-mca is a tool within the llvm suite that can be used to statically measure the performance of machine code for a specific CPU.

$LLVM_BUILD_DIR/bin/llvm-mca -mtriple=riscv64 -mcpu=sifive-p550 pi.s

Around the area where there should have been an fdiv.d, there were two fdiv.s like GCC. From this, I concluded that the fdiv.d instruction in the loop must have been a recent regression. LLVM used to be able to narrow the double to a float, but the latest builds can no longer convert the double into a float.

I confirmed this was indeed a regression by comparing LLVM to a prior build for the same CPU and benchmark.

https://cc-perf.igalia.com/db_default/v4/nts/profile/260/406/4

Below is the assembly generated by the prior build of LLVM.

No fdiv.d instruction on the prior build. In its place, an fdiv.s is used.

Out-of-Order Execution

If you are wondering as to why the ordering of the assembly above and the assembly from the new LLVM build look different, it’s because the target CPU, the SiFive P550, is an out of order CPU. Unlike the CPU in the Banana Pi mentioned in the previous post which was an in-order CPU, this target can execute instructions in an order that can produce a higher throughput.

I’m not exactly sure why the fdiv.d is higher, but my suspicions would be that since the double division has a significantly higher latency, the CPU is trying to dispatch other instructions to ‘hide’ this latency. The fcvt.s.d instruction that consumes the value of the fdiv.d, ft1, would need to wait for 33 cycles, so perhaps the CPU made the decision to schedule instructions between them.

Where is this happening?

At this point in time we don’t know why this happened but a step in the right direction would be to figure out where it happens. Perhaps it was some change in the RISCV backend? The command below gives us the commits related to RISC-V.

git log --after="2026-04-01 00:11" --before="2026-04-04 00:10" | grep -E "RISC"

[RISCV] Select add(vec, splat(scalar)) to PADD_*S for P extension (#190303)
[RISCV] Allow coalesceVSETVLIs to move an LI if it allows a vsetvli to be mutated. (#190287)
[RISCV][TTI] Update cost and prevent exceed m8 for vector.extract.last.active (#188160)
[RISCV] Check EnsureWholeVectorRegisterMoveValidVTYPE in RISCVInsertVSETVLI::transferBefore. (#190022)
[RISCV] Remove codegen for vp_ctlz, vp_cttz, vp_ctpop (#189904)
Part of the work to remove trivial VP intrinsics from the RISC-V
[RISCV] Move unpaired instruction back in RISCVLoadStoreOptimizer (#189912)
`RISCVLoadStoreOptimizer` moves the instruction adjacent to the other
[RISCV] Fix stackmap shadow trimming NOP size for compressed targets (#189774)
[RISCV] Relax VL constraint in convertSameMaskVMergeToVMv (#189797)
[RISCV] Add SATI_RV64/USATI_RV64 to RISCVOptWInstrs. (#190030)
and the RISCVISD::SATI encoding uses the type width minus one.
[RISCV][MCA] Update sifive-p670 tests to consume input files instead (#189785)
[RISCV] Remove codegen for vp_minnum, vp_maxnum (#189899)
Part of the work to remove trivial VP intrinsics from the RISC-V
[RISCV] Add RISCVISD::USATI/SATI to computeKnownBitsForTargetNode/ComputeNumSignBitsForTargetNode. (#189702)
[RISCV] Add assertions to VSETVLIInfo::hasSEWLMULRatioOnly(). NFC (#189799)
[RISCV] combine-is_fpclass.ll - add initial tests showing failure to constant fold ISD::IS_FPCLASS nodes (#189940)
[RISCV] Remove codegen for VP float rounding intrinsics (#189896)
Part of the work to remove trivial VP intrinsics from the RISC-V
[RISCV] Remove codegen for vp_lrint, vp_llrint (#189714)
Part of the work to remove trivial VP intrinsics from the RISC-V
[RISCV] Add codegen support for SATI and USATI. (#189532)

None of these commits immediately stood out to me, so I put them off. Investigating the middle-end, I decided to look at the final LLVM IR produced by opt with my local LLVM version that was a few days old. Remember that my build is the ‘working’ build. Here is the LLVM IR produced at the beginning of the pipeline (right after clang), and at the end.

Gets the LLVM IR right after the pipeline:

$LLVM_BUILD_DIR/bin/clang -O3 \
--target=riscv64-unknown-linux-gnu \
-march=rv64gc_zba_zbb \
--sysroot=/usr/riscv64-linux-gnu \
-Xclang -disable-llvm-passes \
-S -emit-llvm pi.c -o pi_raw_old.ll

Gets the LLVM IR at the end of the optimization pipeline:

$LLVM_BUILD_DIR/bin/clang -O3 \
  --target=riscv64-unknown-linux-gnu \
  -march=rv64gc_zba_zbb \
  --sysroot=/usr/riscv64-linux-gnu \
  -S -emit-llvm pi.c -o pi.ll

    graph LR
    %% Main Compiler Flow
    Source[Source Code<br/><b>pi.c</b>] --> FE[<b>Front-End</b><br/><i>Clang/Lexer/Parser</i>]
    
    subgraph MiddleEnd ["<b>Middle-End</b> (Optimization Pipeline)"]
        direction LR
        ME_Start(<b>Beginning of Middle-End</b><br/><i>Unoptimized IR</i>) --> Opts[<i>InstCombine, LoopUnroll, GVN, etc.</i>]
        Opts --> ME_End(<b>End of Middle-End</b><br/><i>Optimized IR</i>)
    end
    
    FE --> ME_Start
    ME_End --> BE[<b>Backend</b><br/><i>Code Generator/Instruction Selection</i>]
    BE --> Asm(<b>Assembly</b><br/><i>RISC-V</i>)

    %% Styling Main Pipeline
    style Source fill:none,stroke:#888,stroke-width:1px
    style FE fill:none,stroke:#888,stroke-width:1px
    style ME_Start fill:none,stroke:#1e88e5,stroke-width:2px
    style ME_End fill:none,stroke:#1e88e5,stroke-width:2px
    style Opts fill:none,stroke:#888,stroke-width:1px,stroke-dasharray: 5 5
    style BE fill:none,stroke:#888,stroke-width:1px
    style Asm fill:none,stroke:#888,stroke-width:1px
    style MiddleEnd fill:none,stroke:#444,stroke-width:1px,stroke-dasharray: 3 3

    %% Compact Command Boxes
    %% We use <div> and <code> to keep things tight and prevent block-level gaps
    Cmd_Raw["<div style='line-height:1.2;'><b>Command 1: Raw IR</b><br/><code style='background:none;color:inherit;'>-Xclang -disable-llvm-passes</code><br/><small>Produces <b>pi_raw_old.ll</b></small></div>"]
    
    Cmd_Opt["<div style='line-height:1.2;'><b>Command 2: Optimized IR</b><br/><code style='background:none;color:inherit;'>-O3 -S -emit-llvm</code><br/><small>Produces <b>pi.ll</b></small></div>"]

    %% Connections
    Cmd_Raw -.-> ME_Start
    Cmd_Opt -.-> ME_End

    %% Style Command Notes
    style Cmd_Raw fill:#fff3e011,stroke:#f57c00,stroke-width:1px,rx:10,ry:10
    style Cmd_Opt fill:#fff3e011,stroke:#f57c00,stroke-width:1px,rx:10,ry:10

If you are confused as to what I am doing, hopefully the diagram above illustrates this better. If the middle-end is responsible for narrowing the double to a float, I am trying to get an idea of what optimizations are happening to the IR that causes this.

Instead of doing this, you could also use the print-before and print-after on a specific pass to see what the pass is doing to the IR.

This is the relevant snippet of the IR from the beginning of the optimization pipeline.

%conv  = sitofp i64 %5 to float        ; int -> float
%conv2 = fpext float %conv to double   ; float -> double 
%div3  = fdiv double %conv2, 7.438300e+04  ; fdiv.d 
%conv4 = fptrunc double %div3 to float ; double -> float

Which was converted into the following by the end of the pipeline.

%conv  = uitofp nneg i64 %0 to float   ; int -> float 
%conv4 = fdiv float %conv, 7.438300e+04 ; fdiv.s

We can see that by the end of the middle-end, the double was narrowed to a float. I hope it’s clear from these snippets that it was the middle-end responsible for narrowing this initial double value to a float.

If you’re confused as to why these seemingly redundant cast operations are produced in the first place, taking a cursory glance at the source code can help.

Source

int main(int argc, char *argv[]) {
  float ztot, yran, ymult, ymod, x, y, z, pi, prod;
  long int low, ixran, itot, j, iprod;

  ...

  for(j=1; j<=itot; j++) {
    iprod = 27611 * ixran;
    ixran = iprod - 74383*(long int)(iprod/74383);
    x = (float)ixran / 74383.0;

    ...

  }
    
  ...

}

This specific line shows that 74383.0 is a double in the source code.

    x = (float)ixran / 74383.0;

This is why the LLVM IR has an fpext converting a float to a double, and then a fptrunc of the double back to a float.

This may be obvious, but I’ll state it regardless. If the literal above had an initially been a float like below, the fdiv.d would have never been produced.

    x = (float)ixran / 74383.0f;

Given a sufficient optimization level, the compiler should still be able to catch something like this, but it’s still cool seeing how such a little change in code can have such a huge difference. In this case, over +19% in cycles!

The table below shows the mapping of LLVM IR to the C source code, as well as a brief note on what the specified IR does.

LLVM IR	C Source	Notes
`%mul = mul nuw nsw i64 %ixran.053, 27611`	`iprod = 27611 * ixran`	integer multiply
`%0 = urem i64 %mul, 74383`	`ixran = iprod - 74383*(long int)(iprod/74383)`	compiler optimized modulo via `urem`
`%conv2 = uitofp nneg i64 %0 to double`	`(float)ixran`	cast to double first due to `74383.0` being a double literal
`%div3 = fdiv double %conv2, 7.438300e+04`	`/ 74383.0`	division in double precision because `74383.0` is a double literal in C
`%conv4 = fptrunc double %div3 to float`	`x = (float)...`	explicit `(float)` cast truncates result back to float

The LLVM build at the time was producing the following.

%mul = mul nuw nsw i64 %ixran.053, 27611
%0 = urem i64 %mul, 74383
%conv2 = uitofp nneg i64 %0 to double
%div3 = fdiv double %conv2, 7.438300e+04
%conv4 = fptrunc double %div3 to float

We can see that the operands of the div3 operations is %conv2, and a decimal value. %conv2 is the result of a cast operation converting %0 into a double, but the maximum value of %0, 74383, can fit within a float.

Looking at the result from llvm-mca, we can see the following for fdiv.d.

 1      33    32.00                       fdiv.d	ft1, ft1, fa3

This shows that fdiv.d has a latency of 33 cycles, significantly longer than fdiv.s’s 19 cycles. In our performance comparison, the older LLVM build reported a Reciprocal Throughput (RThroughput) of 86.0, whereas the newer build has increased to 100.0. RThroughput represents the number of clock cycles the processor must wait before it can start executing another instruction of the same type. So the lower, the better.

This shows how a double division is pretty expensive compared to a float division.

From the list of commits in those past few days, the one below immediately stood out to me.

[InstCombine] Use ComputeNumSignBits in isKnownExactCastIntToFP (#190235)

For signed int-to-FP casts, ComputeNumSignBits can prove exactness where
computeKnownBits cannot -- e.g. through ashr(shl x, a), b where sign propagation is
tracked precisely but individual known bits are all unknown.

InstCombine is an LLVM middle-end optimization pass that combines adjacent or related instructions into single, more efficient operations. It’s broad in its duties, so examples of InstCombine optimizations can vary. For example, InstCombine can reduce x = x * 2 to x = x << 1;

Given that the newest build can no longer cast that integer to a float, this commit message with the int-to-FP casts (sitofp/uitofp) immediately roused my suspicions, and they were confirmed when I built LLVM before and after that commit. Before this change, everything worked and afterward, fdiv.d instructions appeared in the assembly.

I should also note that this patch is an improvement - the InstCombiner pass has more information available to it - but sometimes improvements can cause regression(s) elsewhere in unforeseeable ways. I would posit this gives opportunities for folks like me to contribute 😅

Why is this happening?

Looking at the diff on this commit, I can see that that the author of the patch added known bit analysis to the function isKnownExactCastIntToFP.

static bool isKnownExactCastIntToFP(CastInst &I, InstCombinerImpl &IC) {
  
  ...

  // For sitofp, the sign maps to the FP sign bit, so only magnitude bits
  // (BitWidth - NumSignBits) consume mantissa.
  if (IsSigned) {
    SigBits =
        (int)SrcTy->getScalarSizeInBits() - IC.ComputeNumSignBits(Src, &I);
    if (SigBits <= DestNumSigBits)
      return true;
  }

  return false;
}

You don’t need an in-depth understanding of what this changed did, but we can infer that this change is causing isKnownExactCastIntToFP to return true, when it was returning false before. So I decided to look at all the call sites of isKnownExactCastIntToFP to gain a better understanding of how this could lead to a regression.

The following function calls isKnownExactCastIntToFP. It is called to reduce an fpext instruction if a preceding instruction casts an integer to FP like so: itofp i64 x to float -> fpext float x to double. This can just be reduced to a itofp i64 x to double instead.

Instruction *InstCombinerImpl::visitFPExt(CastInst &FPExt) {
  // If the source operand is a cast from integer to FP and known exact, then
  // cast the integer operand directly to the destination type.
  Type *Ty = FPExt.getType();
  Value *Src = FPExt.getOperand(0);
  if (isa<SIToFPInst>(Src) || isa<UIToFPInst>(Src)) {
    auto *FPCast = cast<CastInst>(Src);
    if (isKnownExactCastIntToFP(*FPCast))
      return CastInst::Create(FPCast->getOpcode(), FPCast->getOperand(0), Ty);
  }

  return commonCastTransforms(FPExt);
}

And this puts it together.

Referring the LLVM IR that was generated at the beginning of the pipeline.

%conv  = sitofp i64 %5 to float        ; int -> float
%conv2 = fpext float %conv to double   ; float -> double 
%div3  = fdiv double %conv2, 7.438300e+04  ; fdiv.d 
%conv4 = fptrunc double %div3 to float ; double -> float

And the LLVM IR at the end of the pipeline.

%conv2 = uitofp nneg i64 %0 to double
%div3 = fdiv double %conv2, 7.438300e+04
%conv4 = fptrunc double %div3 to float

    graph TD
    %% Node Definitions
    N1["%conv = sitofp i64 %5 to float<br/><i>int -> float</i>"]
    N2["%conv2 = fpext float %conv to double<br/><i>float -> double</i>"]
    N3["%div3 = fdiv double %conv2, 7.438300e+04<br/><i>fdiv.d</i>"]
    N4["%conv4 = fptrunc double %div3 to float<br/><i>double -> float</i>"]

    %% Data Flow
    N1 --> N2
    N2 --> N3
    N3 --> N4

    %% Subgraph
    subgraph CastGroup ["Initial Casts"]
        direction TB
        N1
        N2
    end

    %% Annotation Node
    AnnotateNode["<b>InstCombinerCast</b><br/>Reduces this pattern to %conv2 = uitofp nneg i64 %0 to double"]
    
    %% Styling - Using neutral strokes that work in both modes
    style AnnotateNode fill:#0288d122,stroke:#0288d1,stroke-width:2px,rx:10,ry:10
    style CastGroup fill:none,stroke:#888,stroke-dasharray: 5 5

    %% Invisible edges for layout
    AnnotateNode -.-> N1
    AnnotateNode -.-> N2

The InstCombiner pass after that patch is able to optimize the sitofp i64 to float, followed by fpext float to double by reducing it a single uitofp nneg i64 %0 to double.

However, visitFPTrunc optimized the following pattern as commented in the code:

// If we have fptrunc(OpI (fpextend x), (fpextend y)), we would like to

// simplify this expression to avoid one or more of the trunc/extend

// operations if we can do so without changing the numerical results.

//

// The exact manner in which the widths of the operands interact to limit

// what we can and cannot do safely varies from operation to operation, and

// is explained below in the various case statements.

The optimized LLVM IR now no longer has that fpext instruction, so the InstCombiner pass and specifically visitFPTrunc can no longer narrow the double to a float.

Landing a solution

So we diagnosed the issue - the recent patch to InstCombine, which improved the logic, causes the pass to no longer be able to perform another optimization. We need to teach the InstCombiner to narrow an operation earlier with the uitofp/sitofp if there’s an fptrunc later anyways.

    graph TD
    %% Define the Instruction Spine
    subgraph Flow ["LLVM Data Flow"]
        direction TB
        N0["%0 = urem i64 %mul, 74383<br/><i>Range Provider: 0 to 74,382</i>"]
        N1["%conv2 = uitofp nneg i64 %0 to double<br/><i>Integer to Double</i>"]
        N2["%div3 = fdiv double %conv2, 7.438300e+04<br/><i>Double Division</i>"]
        N3["%conv4 = fptrunc double %div3 to float<br/><i>Double to Float</i>"]

        %% Define the vertical chain
        N0 --> N1
        N1 --> N2
        N2 --> N3
    end

    %% Define the Optimizer Node on the side
    IC["<b>InstCombiner</b><br/>Reduces to float-precision<br/>math if inputs fit."]
    
    %% Styling: Using #AARRGGBB or #RRGGBBAA logic for Hextra
    %% fill:#1b5e2022 is a very transparent green (approx 13% opacity)
    style IC fill:#1b5e2022,stroke:#4caf50,stroke-width:2px,rx:10,ry:10
    
    %% Subgraph styling to keep it neutral
    style Flow fill:none,stroke:#888888,stroke-dasharray: 5 5

    %% Dotted arrow connections
    IC -.-> N0
    IC -.-> N1
    IC -.-> N3

I first raised an issue on Github. While I was confident that this was a valid issue, I wanted to double check that this warranted a fix. I do want to thank the author of that patch I mentioned earlier, @SavchenkoValeriy, as he was able to offer me guidance on solving this issue, as well as providing reviews to my PR. My very initial solution would’ve been pretty convoluted but he was able to offer me a much simpler approach.

GitHub Issue: https://github.com/llvm/llvm-project/issues/190503

PR: https://github.com/llvm/llvm-project/pull/190550

As pointed out by my reviewer, the current issue with isKnownExactCastIntToFP currently only checks if the casting instruction CastInst.

  %0 = urem i64 %mul, 74383
  %conv2 = uitofp nneg i64 %0 to double
  %div3 = fdiv double %conv2, 7.438300e+04
  %conv4 = fptrunc double %div3 to float

The code below is the visitFPTrunc function You can see the switch statement with the different operations, including the FDiv, and this corresponds to the fdiv.d/fdiv.s. FPT corresponds to the fptrunc instruction, BO corresponds to FPT.getOperand(0), so it would refer to %conv2. We need to see if we can instead convert this uitofp into a float, so we need to modify getMinimumFPType to check for cast operations as well as fpext instructions.

Instruction *InstCombinerImpl::visitFPTrunc(FPTruncInst &FPT) {
  if (Instruction *I = commonCastTransforms(FPT))
    return I;
  
  ...

  Type *Ty = FPT.getType();
  auto *BO = dyn_cast<BinaryOperator>(FPT.getOperand(0));
  if (BO && BO->hasOneUse()) {
    Type *LHSMinType = getMinimumFPType(BO->getOperand(0), PreferBFloat);
    Type *RHSMinType = getMinimumFPType(BO->getOperand(1), PreferBFloat);

    switch (BO->getOpcode()) {
      default: break;
      case Instruction::FAdd:
      case Instruction::FSub:
      ...
  ...

}

One of my initial idea was to modify isKnownExactCastIntToFP to accept a parameter with a different Type (f32 in my case) but with a default nullptr value. This would allow us to modify just the header definition and its implementation. My reviewer instead proposed the idea of making a variant of isKnownExactCastIntToFP, canBeCastedExactlyIntToFP. This will do the actual analysis with the type given to it, and isKnownExactCastIntToFP can call it. I mention this to show that it’s good to interact with the community and seek ideas from others. They can come up with ideas which for a variety of reasons could be better.

Below is the final git diff. We split isKnownExactCastIntToFP and created canBeCastedExactlyIntToFP to perform the actual analysis, and isKnownExactCastIntToFP call it. Then, we have getMinimumFPType call canBeCastedExactlyIntToFP.

index bc52bf1168d4..2688891c1509 100644
--- a/llvm/include/llvm/Transforms/InstCombine/InstCombiner.h
+++ b/llvm/include/llvm/Transforms/InstCombine/InstCombiner.h
@@ -481,6 +481,8 @@ public:
   /// Return true if the cast from integer to FP can be proven to be exact
   /// for all possible inputs (the conversion does not lose any precision).
   bool isKnownExactCastIntToFP(CastInst &I) const;
+  bool canBeCastedExactlyIntToFP(Value *V, Type *FPTy, bool IsSigned,
+                                 const Instruction *CxtI = nullptr) const;

   OverflowResult computeOverflowForUnsignedMul(const Value *LHS,
                                                const Value *RHS,
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp b/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp
index e3c39a3c193e..0cd035bd1413 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp
@@ -2039,10 +2039,17 @@ static Type *shrinkFPConstantVector(Value *V, bool PreferBFloat) {
 }

 /// Find the minimum FP type we can safely truncate to.
-static Type *getMinimumFPType(Value *V, bool PreferBFloat) {
+static Type *getMinimumFPType(Value *V, Type *PreferredTy, InstCombiner &IC) {
   if (auto *FPExt = dyn_cast<FPExtInst>(V))
     return FPExt->getOperand(0)->getType();

+  Value *Src;
+  if (match(V, m_IToFP(m_Value(Src))) &&
+      IC.canBeCastedExactlyIntToFP(Src, PreferredTy, isa<SIToFPInst>(V),
+                                   cast<Instruction>(V)))
+    return PreferredTy;
+
+  bool PreferBFloat = PreferredTy->getScalarType()->isBFloatTy();

Notice how we now call canBeCastedExactlyIntToFP with the Type of the fptrunc instruction being passed in. Src in this case is the input to BO->getOperand(0), and BO is the 0th operand of the fptrunc instruction. Remember the llvm ir earlier:

%0 = urem i64 %mul, 74383
%conv2 = uitofp nneg i64 %0 to double
%div3 = fdiv double %conv2, 7.438300e+04
%conv4 = fptrunc double %div3 to float

The fptrunc is converting the input into a float, so the type Ty is f32. BO is %div3, BO->getOperand(0) is %conv2 and the m_IToFP(m_Value(Src)) puts the input of BO->getOperand(0) into Src. The table below shows the mapping of the visitFPTrunc code to the LLVM IR variables.

`visitFPTrunc` Variables	LLVM IR Variables	Value
`BO`	`%div3`	`fdiv double %conv2, 7.438300e+04`
`BO->getOperand(0)`	`%conv2`	`uitofp nneg i64 %0 to double`
`BO->getOperand(1)`	`7.438300e+04`	double constant
`Src`	`%0`	`urem i64 %mul, 74383`

Result

Did this work? By running the command below we can see the LLVM IR after my patch.

$LLVM_BUILD_DIR/bin/clang -O3 \
  --target=riscv64-unknown-linux-gnu \
  -march=rv64gc_zba_zbb \
  --sysroot=/usr/riscv64-linux-gnu \
  -S -emit-llvm pi.c -o pi_fixed.ll

And this is the relevant LLVM IR.

%mul = mul nuw nsw i64 %ixran.053, 27611
%0 = urem i64 %mul, 74383
%1 = uitofp nneg i64 %0 to float
%conv4 = fdiv float %1, 7.438300e+04

It worked! 🥹

The fptrunc is gone, and the cast operation uitofp now converts it into a float.

Shown below are the results after my patch was merged. We can see that the target can execute the benchmark in 1.67 Bn cycles, about a 25% improvement.

https://cc-perf.igalia.com/db_default/v4/nts/profile/260/426/422

Analysing a Benchmark on LLVM RISCV

Sun, 05 Apr 2026 18:14:45 -0400

This was my attempt at improving a benchmark on LLVM RISCV. Unfortunately, the solution to this problem is kind of out of my league. But the attempt does make for a good blog post.

Introduction

Recently, I found this blog post by Luke Lau, in which he shows how he was able to make improvements to the middle-end (and ultimately the generated code) in LLVM, resulting in better performance for RISC-V targets on LLVM. He does this by comparing the assembly of LLVM and GCC with the llvm test suite and analysing the cases where LLVM falls behind.

I found this whole process of looking at assembly and figuring out where the root issue to be very interesting, and this blog goes over my first thorough attempt at this.

I should also note that the company Igalia hosts these benchmarks, and runs periodic updates. You can take a look and see how LLVM compares to GCC on certain RISCV targets.

Code Analysis

This link will take you to the benchmark, but I have pasted the relevant assembly in the sections below, as well as the relevant source code. Another thing to note is that the CPU used in this benchmark is the Banana Pi F3, a 64-bit in-order CPU with RISCV’s vector extension, RVV.

I will go into a level of detail that’s probably not necessary, but as I’m still new to assembly, I found understanding what each instruction to be helpful. If you’re already familiar with assembly or not really interested in the nitty gritty, feel free to skim over this section.

Source

for (k=0; k<1000; k++) {
  for (i = n-1; i >= 0; i--) {
    y[i] += x[i];
  }
}

Note

The inner loop here does not have any dependency on the outer loop (as in k is not use as a subscript in the inner loop).

This is the results of the benchmark.

LLVM


BB1: 
li	a3, 0x0      	
mv	a4, s0

BB2: 
vsetvli	a5, a4, e32, m2, ta, ma      	
not	a2, a3      	
add	a2, a2, s0      	
sub	a1, t0, a5      	
sh2add	a0, a2, s2      	
sh2add	a2, a2, s1      	
sh2add	a0, a1, a0      	
sh2add	a1, a1, a2      	
vle32.v	v8, (a0)      	
vle32.v	v10, (a1)      	
sub	a4, a4, a5      	
vadd.vv	v8, v10, v8      	
vse32.v	v8, (a1)      	
add	a3, a3, a5      	
bnez	a4, 0x7b2 <main+0x7a>

BB3:
addiw	a7, a7, 0x1      	
bne	a7, a6, 0x7ae <main+0x76>

    graph TD
    BB1 --> BB2
    BB2 --> BB3
    BB3 --> BB2
    BB2 --> BB2

Info

A Basic Block is a sequence of instructions in a program that executes linearly, meaning it has exactly one entry point at the start and one exit point at the end. Because there are no internal jumps or branches, once the first instruction is triggered, the entire block is guaranteed to execute in order.

This is a bit of a doozy, but it’s not too bad when you match it with the source code. The middle basic block is basically doing the third line of the code snippet provided.

y[i] += x[i];

The four sh2add instructions are getting the location of the addresses of y[i] and x[i]. They perform the following logic:

\[ Op0 = Op1 \times 4 + Op2 \]

In this case, Op2 is the base address and Op1 is the index i or induction variable, and multiplying by four (an integer is 4 bytes), will get us the correct address. We see four of these instructions since there’s two arrays and the inner for loop is decrementing in value.

Notice how third sh2add is writing it’s result into a0, and how the result of the second sh2add is just needed as an operand for the fourth sh2add. Essentially, the registers to take note of are a0, and a1, as these are where the addresses were stored in.

After that, we can see the two vector load instructions vle32.v, and a vector store instruction later vse32.v. At this point, you may be able to surmise that LLVM was able to vectorize this loop, as in was able to use the RVV extension of the BPI-F3 cpu to operate on multiple elements per iteration. Pretty cool!

GCC

BB1:
lw	a3,0(a1)	
lw	a4,0(a2)	
li	a5,500	
slliw	a3,a3,0x1

BB2: 
addiw	a5,a5,-1	
addw	a4,a4,a3	
bnez	a5,1064e <main+0x8e

BB3:
sw	a4,0(a2)	
addi	a2,a2,-4	
addi	a1,a1,-4	
bne	a2,a0,10642 <main+0x82>

    graph TD
    BB1 --> BB2
    BB2 --> BB3
    BB3 --> BB2
    BB2 --> BB2

What GCC did here was very clever. Essentially, GCC’s middle-end performed what’s called a Loop Interchange. It swapped the inner loop for the outer loop, resulting in something would look like the following in code.

for (i = n-1; i >= 0; i--) {
  for (k=0; k<1000; k++) {
    y[i] += x[i];
  }
}

Why is this better? Take a look at the CFG and assembly and notice how the second basic block, BB2, in both the GCC assembly and LLVM points to itself. It’s more obvious if you look at the website, as it clearly shows those basic blocks as hotspots, in which the program spends most of its cycles on these basic blocks. In this case, both programs spend 99% of their cycles on BB2.

By doing the interchange, GCC has only add operations in BB2, while LLVM has two loads and a store. Meaning that when Banana Pi runs the code produced by clang, it will get a lot of cache misses and stalls. If you look at the image of the benchmark results, you can see that despite LLVM issuing less instructions, it still takes several more cycles. This is because the CPU has a limited size cache, and if the memory it’s looking for is not in cache, it has to go down the cache hierarchy to look for it, and each tier of memory is significantly slower.

I found it pretty cool to visually see the direct impact your code can have on the performance, and how compilers can significantly improve performance. In this case, just reordering the loop can significantly increase the performance.

Loop Interchange

Ok great, the solution is to simply run loop interchange on LLVM as well. Unfortunately, the loop interchange is not enabled by default. I won’t even bother to summarize why as it seems quite complicated, but take a look at the discourse if you’re curious. Suffice to say, it seems like there’s still work be done on the pass.

But I was curious - I wanted to take a look at the LLVM IR of the interchanged loop. I ran the following command to get emit the LLVM IR before the Loop Vectorizer pass.

../bin/clang --target=riscv64-linux-gnu \
    -march=rva22u64_v -mcpu=spacemit-x60 -O3 \
    --sysroot=/usr/riscv64-linux-gnu \
    -mllvm -print-before=loop-vectorize \
    -mllvm -print-module-scope \
    -S -emit-llvm ary3.c -o ary3_prevec.ll

Then I ran the following command to get the LLVM IR, as well as the debug output, of the loop interchange pass and DependenceAnalysis, which supports the Loop interchange pass.

../bin/opt -passes='loop-interchange' \
    -mtriple=riscv64 -mattr=+v \
    -debug-only=loop-interchange,da \
    -S ary3_prevec.ll -o ary3_interchange.ll 2> interchange_log.txt

This ultimately did not change the LLVM IR, indicating that loop interchange did not occur.

Troubleshooting

Taking a look at the debug message, I noticed this excerpt which proved to be key in understanding why it did not work. By the way, for.bod14.us is the name of the basic block for the inner for loop, equivalent to BB2 above.

Dependency matrix before interchange:
* =
... 
Failed interchange InnerLoopId = 1 and OuterLoopId = 0 due to dependence
Cannot prove legality, not interchanging loops 'for.cond11.preheader.us' and 'for.body14.us'

I’ve listed some code from LoopInterchange.cpp that should explain why the pass failed to run.

 for (unsigned II = 1; II <= Levels; ++II) {
  // `DVEntry::LE` is converted to `*`. This is because `LE` means `<`
  // or `=`, for which we don't have an equivalent representation, so
  // that the conservative approximation is necessary. The same goes for
  // `DVEntry::GE`.
  // TODO: Use of fine-grained expressions allows for more accurate
  // analysis.
  unsigned Dir = D->getDirection(II);
  if (Dir == Dependence::DVEntry::LT)
    Direction = '<';
  else if (Dir == Dependence::DVEntry::GT)
    Direction = '>';
  else if (Dir == Dependence::DVEntry::EQ)
    Direction = '=';
  else
    Direction = '*';
  Dep.push_back(Direction);
 }

And this code, which is called during the legality check.

static std::optional<bool>
isLexicographicallyPositive(ArrayRef<char> DV, unsigned Begin, unsigned End) {
  for (unsigned char Direction : DV.slice(Begin, End - Begin)) {
    if (Direction == '<')
      return true;
    if (Direction == '>' || Direction == '*')
      return false;
  }
  return std::nullopt;
}

Basically, the nested for loop in the source code was designated with a dependency matrix of * =, which each character representing the dependency direction of each loop level. So the outer loop has a dependency direction of * and the inner loop has a dependency direction of =. I’m still don’t fully understand all the terminology used in this pass. Take a look at my discourse post where a user, @kasuga-fj, was kind enough to break this down for me.

I’ve not checked the details, but at a glance, I think the dependency direction for the outer loop should be DVEntry::ALL. The direction represents the possible relationships between the values of k when the same memory location is accessed. More precisely, in this case, it answers this question: when regarding the access to y as a function f(k, i), what are the possible relations between k0 and k1 when f(k0, i) = f(k1, i) for some i?

Here, f(k, i) = i. So, for example, f(0, 0) = f(1, 0) = … = f(999, 0). Thus the outer loop has a dependency with all directions.

…

I believe the right direction is to allow swapping the adjacent loops if the direction vector is * = or = *. But this is also non-trivial in terms of soundness, and we need to think this through a bit more…

His proposed solution is to allow the swapping of the dependency matrix given it’s *=. As shown in isLexicographicallyPositive, this check fails since it’s specifically looks for < character as the leftmost character. I’m guessing this hasn’t been implemented yet since it would probably cause a lot of regressions and/or would require more discussion among the community.

Conclusion

All in all, this process was pretty rewarding. I couldn’t land a change to improve or fix this, nor did I did discover a missing case, but I do think this project was still worth it.

My intro to the RISCV LLVM Backend: modifying an existing LLVM pass to merge GPRPair moves

Fri, 27 Feb 2026 18:02:45 -0500

For a while now I’ve been interested in the RISCV architecture, and recently I had the opportunity to indulge this interest by working on a pass improving code gen on the RISCV backend for LLVM.

Credits to @lenary to raising this issue.

Relevant PRs:

Some context

There exists a pass known as the RISCVMoveMerger. Prior to my work this pass was solely intended to merge particular instructions on RISCV targets with the zcmp or xqccmp extension. An extension can be thought of as an addition to the existing RISCV ISA.

In assembly, we have instructions that move values between registers, referred to as move or copy instructions. In a 64-bit CPU, the registers would be 64 bits wide, and correspondingly, the registers would be 32-bits in a 32-bit CPU. A good chunk of variables in C/C++ are interpreted as 32-bits unless specified (int, float, etc.). So what happens when a code specifies a 64-bit integer on a 32-bit CPU? Well, something akin to the following. mv indicates this is a move instruction, the first operand is the destination, and the second operand is the source.

// Copy the lower 32 bits
mv a0, t0    
// Copy the upper 32 bits
mv a1, t1

In the context of the RISCVMoveMerger, the zcmp extension has two instructions of concern. They are CM.MVSA01 and CM.MVA01S. They use the following syntax.

mv a0, r1
mv a1, r2
// Statement below is equivalent.
// We are moving values in register r1 and r2 into a0 and a1 respectively.
cm.mvsa01 r1, r2

mv r1, a0 
mv r2, a1 
// Statement below is equivalent.
// We are moving values in register a0 and a1 into r1 and r2 respectively.
cm.mva01s r1, r2

a0 is designated as the first argument register. Whenever a function is called, it’s input parameters will be placed in a0. For example, when calling foo(int x)i, x would be stored in a0. These compressed instructions take advantage of this by making the source and destination implicit. The cm* don’t implicitly define the source or destination registers, and instead implicitly use a0/a1.

To get back to RISCVMoveMerger, the goal of this pass was to find instructions like these two 32-bit moves:

mv a0, r1         mv r1, a0
mv a1, r2         mb r2, a1

and merge them to make use of zcmp’s cm.mv* instructions. It can be thought of as providing a “macro” to handle 64-bit logic in a single step.

My PR(s)

Hopefully that laid the ground work for what the RISCVMoveMerger was doing prior to my work. I added onto this pass by extending to other types of possible 32-bit to 64-bit pairs for other targets. This was a task raised by @lenary, who allowed me to pursue an implementation.

RISCV supports quite a lot of different extension, and two extensions that concerns this pass is the experimental-p and zdinx target. They respectively support the instructions addd, and fsgnj.d which both can support GPRPairs as operands. A GPRPair is a pair of registers which can be treated as one unit. We can take a look at RISCVRegisterInfo.td - a tablegen file that describes the register info the RISCV architecture - to find out the exact details of GPRPairs.

def GPRPair : RISCVRegisterClass<[XLenPairVT, XLenPairFVT], 64, (add
    X10_X11, X12_X13, X14_X15, X16_X17,
    X6_X7,
    X28_X29, X30_X31,
    X8_X9,
    X18_X19, X20_X21, X22_X23, X24_X25, X26_X27,
    X0_Pair, X2_X3, X4_X5
)>;

I hope this is starting to come together. RISCVMoveMerger can be additionally used to find two move instructions such that they can be merged into one instruction.

mv a0, s0 
mv a1, s1

// or 
mv a1, s1
mv a0, s0 

// This is the equivalent for zdinx.
fmv.d a0, s0

// This is the equivalent for experimental-p. Note that a third operand is 
// used but this just adds zero to s0, so it's still a move.
addd a0, s0, zero

Honestly at this point, I basically recounted the major points of my work. Reading further will go into some more minute details which I found interesting.

Current structure & New Functions

This was structure of the pass, and it can be noticed that the current function signatures are tied with finding and merging cm.mv* instructions. For example, MoveFromSToA is a helpful boolean parameter useful for cm.mv* instructions, but useless for GPRPairs.

  bool isCandidateToMergeMVA01S(const DestSourcePair &RegPair);
  bool isCandidateToMergeMVSA01(const DestSourcePair &RegPair);
  // Merge the two instructions indicated into a single pair instruction.
  MachineBasicBlock::iterator
  mergePairedInsns(MachineBasicBlock::iterator I,
                   MachineBasicBlock::iterator Paired, bool MoveFromSToA);
  // Look for C.MV instruction that can be combined with
  // the given instruction into CM.MVA01S or CM.MVSA01. Return the matching
  // instruction if one exists.
  MachineBasicBlock::iterator
  findMatchingInst(MachineBasicBlock::iterator &MBBI, bool MoveFromSToA,
                   const DestSourcePair &RegPair);
  bool mergeMoveSARegPair(MachineBasicBlock &MBB);
  bool runOnMachineFunction(MachineFunction &Fn) override;

The first two isCandidate* bool helpers are used to indicate whether an instruction can be merged into a possible cm.mv instruction. mergePairedInsns is used when two potential instructions are found and the merged. findMatchingInst is used when the first instruction is already found, and it looks for the second pair. mergeMoveSARegPair is used to iterate through instructions in a basic block to initially find the instructions, and call the find and merge functions.

mergeMoveSARegPair can definitely be reused, since it’s goal is to iterate through instructions. However, findMatchingInst and mergeMoveSARegPair are inherently tied to the cm.mv instructions. For GPRPair, there needs to be very similar functions but with a different boolean parameter, EvenRegPair, to indicate if a GPRPair is even or odd, since the logic to make the pair would be slightly different if the pair is even or odd. The snippet below from mergeRegPairInsns may illuminate what I’m trying to convey.

  unsigned GPRPairIdx =
      RegPairIsEven ? RISCV::sub_gpr_even : RISCV::sub_gpr_odd;
  SrcReg1 = TRI->getMatchingSuperReg(FirstPair.Source->getReg(), GPRPairIdx,
                                     &RISCV::GPRPairRegClass);
  SrcReg2 = ST->hasStdExtZdinx() ? SrcReg1 : Register(RISCV::X0_Pair);
  DestReg = TRI->getMatchingSuperReg(FirstPair.Destination->getReg(),
                                     GPRPairIdx, &RISCV::GPRPairRegClass);

Some DFA

Dataflow analysis allows compilers to determine how data flows through the program. It allows compilers to make effective and safe decisions about optimizations. It’s essential to understand DFA to work with middle-end and back-end optimizations, and we get to see DFA in action at least within a basic block in this pass. Keep in mind that I did not write the code you’ll see in this section.

Lets think up of a scenario where we found the first potential instruction pair, and the findMatchingInst/Pair is called to iterate through the rest of the basic block to find the second.

mv a0, s0 
ld a2, a1
mv a1, s1

We can see from the snippet above that there could be a potential match here with a0, a1 and s0, s1. However, notice that before the second mv instruction is a ld instruction. This instruction loads the value of a1 into a2. If we were to merge the two move instructions, a1 would get overwritten and the data loaded into a2 would be invalid!

// Say we did merge it so now it's mv a0_a1, s0_s1
mv a0, s0 
// a2 would get the value of s1 instead of the previous (correct) value of a1!
ld a2, a1

The if statement with early return has a pretty good comment explaining it’s purpose and it solves this problem. If there are any instructions between the first instruction, and the second instruction which modifies or uses the destination register of the second instruction, return early.

  for (MachineBasicBlock::iterator I = next_nodbg(MBBI, E); I != E;
       I = next_nodbg(I, E)) {

    MachineInstr &MI = *I;

    if (auto SecondPair = TII->isCopyInstrImpl(MI)) {
      Register SourceReg = SecondPair->Source->getReg();
      Register DestReg = SecondPair->Destination->getReg();

      bool IsCandidate = MoveFromSToA ? isCandidateToMergeMVA01S(*SecondPair)
                                      : isCandidateToMergeMVSA01(*SecondPair);
      if (IsCandidate) {

        ...

        // If paired destination register was modified or used, the source reg
        // was modified, there is no possibility of finding matching
        // instruction so exit early.
        if (!ModifiedRegUnits.available(DestReg) ||
            !UsedRegUnits.available(DestReg) ||
            !ModifiedRegUnits.available(SourceReg))
          return E;

        return I;
      }
    }
    // Update modified / used register units.
    LiveRegUnits::accumulateUsedDefed(MI, ModifiedRegUnits, UsedRegUnits, TRI);
  }

Another DFA we can observe is ensuring the liveness or kill status of registers when moving the second register pair to the first pair’s location. The snippet below demonstrates the problem if the kill status is not considered. Note that the snippet below is an mir file, a lower-level intermediate representation right before the final assembly.

$x10 = COPY $x12               ; (I) The "Target" location for the merge
$x5  = ADD $x13, $x3           ; (It) Middle instruction READS $x13!
$x11 = COPY killed $x13        ; (Paired) The move we want to bring up to Line 1

The third instruction, a move/copy instruction, can be merged with the first instruction. But note that the there is a keyword, killed $x13. It indicates that this register, $x13 is not used after that instruction. If the compiler just moves this to the location of the first instruction and merges it, this would cause an error because it’s still killed but the second instruction reads the value from $x13.

// PairSource is a copy of the second pair. 
// This for loop iterates from the first istruction to the second instruction.
// It checks if any other register reads the register of the second instruction.
// If it does, it sets the kill status to false.
for (auto It = std::next(I); It != Paired && PairedSource.isKill(); ++It)
  if (It->readsRegister(PairedSource.getReg(), TRI))
    PairedSource.setIsKill(false);

Conclusion

Overall, this was a pretty fun PR. It was pretty exciting to work on a PR that was not labelled a ‘Good First Issue’. Some things I want to keep pursuing are issues that in some manner require DFA.

My First LLVM PR

Fri, 21 Nov 2025 13:48:43 -0500

Honestly, it’s actually my second upstreamed PR, but I think it’s worth writing a blog about and the title sounds better then ‘My Second LLVM PR’.

It’s also a revival of the work done by @huhu233, who implemented the feature of this PR for the ARM backend. I also received a lot of help from the PR reviewer, @RKSimon, to ensure the code is up to LLVM’s standards.

Link To PR

What & Why LLVM?

It’s essentially a modular compiler infrastructure that can be used to develop a compiler. It’s the backbone for Clang (C/C++), Rust, Zig (I think they’re migrating though). It’s also the industry standard GPU computing software, like CUDA and HIP. It’s fair to say it’s quite commonplace now in the compiler world.

At a very high level birds-eye overview, LLVM converts the front-end language (C++, rust, CUDA, etc.) into LLVM IR (Intermediate Representation). LLVM IR is sort of its own language, but understanding LLVM IR isn’t necessary to understand the work I have done, so I won’t try to explain it here. The magic of having the intermediate representation is it allows for language-agnostic optimizations that can benefit every front-end.

And the great thing about the llvm community is that they’re quite friendly to newcomers. Most importantly, there’s is a regular supply of Good first issue on the llvm-project, making it easy for me to start contributing.

Brief SelectionDAG Overview

I think this video is a great for beginners to understand how SelectionDAG works. I’ll try my best give a 1 minute crash course below to get a better grasp on what my PR does.

    graph LR
    B[Frontend] -->|LLVM IR| D[opt]
    D -->|LLVM IR| F[llc]
    F --> G[Assembly]
    H[SelectionDAG] -.-> F

From the diagram above, you can see that SelectionDAG is used when converting LLVM IR to the final assembly. It ensures the LLVM IR generated from opt is suitable for the target used. For example, SelectionDAG will perform Type Legalization, where it ensures the type used in the LLVM IR is actually supported on the target such as an i24being promoted to an i32 for X86. This is just an example, and the video linked before goes much into details of the other things SelectionDAG does to ensure legal instructions are generated for the target.

SDValue

The SelectionDAG datastructure is composed of nodes which can represent the actual operations in the IR, such as ADD, LOAD, OR STORE. SDValue represents the output and inputs of these nodes.

MVT

MVT is the union of tpyes that are supported by each targets that uses SelectionDAG. So an MVT could be an i32, f16, v2f32, and the list goes on. You may also encounter an EVT, which is union of the MVT types and all integer, float, and vector types supported by LLVM IR.

FLDEXP & SCALEF

ldexp is a function in C++ which multiplies a floating-point number by an integral power of two.

\[ ldexp(x, n) = x \times 2 ^ n \]

It allows for the precise manipulation of floating-point numbers by directly changing the exponent without touching the mantissa. This avoids rounding errors from repeated multiplication by 2, and ensures exact powers-of-two scaling in binary floating-point

import math
math.ldexp(0.75, 2)  # 0.75 * 2^2 = 3.0

Mantissa & Exponent

A floating-point number represents a value as:

value = mantissa × 2^exponent

Mantissa (significand): holds the significant digits
Exponent: scales the mantissa by a power of 2

Example:
Decimal 6.5 → Binary 110.1₂ → 1.101 × 2²
Mantissa: 1.101, Exponent: 2

\[ Exponent: 2 + 127 = 129_{10} -> {10000001}_2 \]

\[ Mantissa: 101 \]

10000001

10100000000000000000000

Sign | Exponent | Mantissa

Basically. this allow for efficient representation of very large or very small numbers.
Functions like ldexp(x, n) scale the number precisely by powers of 2.

Normally, when such a function gets lowered into assembly, it would generate a lot of assembly since there is no direct mapping. However, the latest X86 CPU have an extension for vector SIMD operations, called AVX-512. Within the AVX-512 extension, is an instruction called VSCALEF that essentially performs ldexp on multiple floating point values with a single instruction. As the name implies, the extension comes with 512-bit registers, allowing the CPU to perform instructions on multiple 64/32-bit elements at once.

AVX512 Feature Subsets

AVX-512 includes a collections of features that a target can include.

AVX512F (Foundation): The base AVX-512 instruction set. Includes 512-bit operations for 32-bit and 64-bit floats/integers, mask registers (k0-k7), and basic operations like VSCALEFPS/VSCALEFPD for 512-bit vectors.
AVX512VL (Vector Length Extensions): Enables AVX-512 instructions on smaller vector sizes (128-bit and 256-bit). Without VLX, AVX-512 instructions only work on 512-bit (ZMM) registers. With VLX, you can use AVX-512 features on XMM (128-bit) and `YMM`` (256-bit) registers.
AVX512FP16 (Half Precision): Adds native support for 16-bit floating-point operations, including VSCALEFPH/VSCALEFSH for half-precision SCALEF operations.

Without VLX, 128-bit and 256-bit vector operations must be widened to 512-bit before using VSCALEF and without FP16, vectors must be widened to f32.

Instruction	Packed/Scalar	Element Type	Widths Supported	Required Features
VSCALEFPD	Packed	FP64	128 / 256 / 512	AVX512F, AVX512VL (for 128/256)
VSCALEFPH	Packed	FP16	128 / 256 / 512	AVX512FP16, AVX512VL (for 128/256)
VSCALEFPS	Packed	FP32	128 / 256 / 512	AVX512F, AVX512VL (for 128/256)
VSCALEFSD	Scalar	FP64	128	AVX512F
VSCALEFSH	Scalar	FP16	128	AVX512FP16
VSCALEFSS	Scalar	FP32	128	AVX512F

LowerFLDEXP

Now to get onto what my PR actually does. On the X86 backend, when the target has the AVX-512 extension, ldexp should get lowered to VSCALEF. As mentioned before, AVX-512 adds 512-bit SIMD registers and vector instructions, but there are also subsets or extensions to AVX-512. The ones relevant to this PR are AVX-512VL and AVX-512FP16. With AVX-512VL, instructions can operate on 128-bit and 256-bit vectors instead of just 512-bit vectors, and AVX-512FP16 allows for instructions to operate on 16-bit floating pointer vectors. The table below displays the vectors this function will lower and the requirements.

Types	Bit Width	Requirements / Notes	Instruction
v16f32	512-bit	- Full 512-bit native vector - No extensions required	VSCALEFPS
v8f64	512-bit	- Full 512-bit native vector - No extensions required	VSCALEFPD
v8f32 / v4f64	256-bit	- Requires AVX512VL (VLX), or - Widening to 512-bit	VSCALEFPS / VSCALEFPD
v4f32 / v2f64	128-bit	- Requires AVX512VL (VLX), or - Widening to 512-bit	VSCALEFPS / VSCALEFPD
v32f16	512-bit	- Requires AVX512FP16	VSCALEFPH
v16f16 / v8f16	128–256-bit	- Requires FP16 + VLX - Requires FP16 + widening to 512-bit (no VLX) - Requires extend f16→f32 and i16→i32 + widening (no FP16 & no VLX)	VSCALEFPH (if FP16)
Scalar f64	64-bit	- Insert into 128-bit vector	VSCALEFSD
Scalar f32	32-bit	- Insert into 128-bit vector	VSCALEFSS
Scalar f16	16-bit	- Requires AVX512FP16, or - Extend f16→f32 and use scalar f32 path	VSCALEFSH (if FP16)

How to call vscalef*

LLVM’s X86 backend uses SelectionDAG nodes to represent target-specific operations during instruction selection. X86ISD::SCALEF and X86ISD::SCALEFS is the X86 backend’s internal opcode for the vscalef and vscalefs (scalar) instruction.

In the code below you will see occurrences of ISD::* and these represent generic DAG opcodes, such as ISD::ADD. X86ISD::* represent DAG nodes specific to the X86 backend.

Difficulties

The biggest hurdle I faced implementing this PR was correctly handling FP16 vectors (v8f16, v16f16) when the AVX512FP16 extension is not available.

The operation requires converting the integer exponent vector (vNi16) into a floating-point exponent vector (vNf16). If the FP16 instruction set isn’t supported, LLVM literally cannot convert an i16 integer into an f16 floating-point number.

Honestly, the solution was pretty simple. Instead of converting the vNi16 to an vNf16, I would sign-extend the Exp vector to an i32 vector.

    graph TD
    subgraph "Before"
        A[v8f16 / v16f16<br/>FP16 Vector] --> B[Extract i16 Exp]
        B --> C[Convert i16 → fp16<br/>❌ NOT SUPPORTED]
        C --> D[fp16 Exponent]
        D --> E[ldexp operation]
    end

    graph TD
    subgraph "After"
        A[v8f16 / v16f16<br/>FP16 Vector] --> B[Widen to v8f32 / v16f32]
        A --> D[Widen to i32 Exp]
        
        B --> E[Widen v8f32 → v16f32]
        D --> F[Widen i32 Exp → v16i32<br/>512-bit vector]
        
        E --> G[ldexp operation<br/>✓ FP32 supported]
        F --> G
    end

As shown below, when the target has VLX and FP16 support, SCALEF can be directly called on v8f16/v16f16 vectors. Without VLX (but with FP16), LowerFLDEXP will widen the vectors to 512-bits (v32f16). Without either, I have to extend both X and Exp to 32 bits, then widen to 512-bits (v16f32).

case MVT::v8f16:
case MVT::v16f16:
    if (Subtarget.hasFP16()) {
      if (Subtarget.hasVLX()) {
        Exp = DAG.getNode(ISD::SINT_TO_FP, DL, XTy, Exp);
        return DAG.getNode(X86ISD::SCALEF, DL, XTy, X, Exp);
      }
      break;
    }
    X = DAG.getFPExtendOrRound(X, DL, XTy.changeVectorElementType(MVT::f32));
    Exp = DAG.getSExtOrTrunc(Exp, DL,
                             X.getSimpleValueType().changeTypeToInteger());
    break;

For v32f16, no prep is needed if it has FP16 support. Otherwise, I would have to split the v32f16 vector into two v16f16 vectors, and recursively call LoewerFLDEXP on the two, concatenating their results.

case MVT::v32f16:
  if (Subtarget.hasFP16()) {
    Exp = DAG.getNode(ISD::SINT_TO_FP, DL, XTy, Exp);
    return DAG.getNode(X86ISD::SCALEF, DL, XTy, X, Exp);
  }
  SDValue Low = DAG.getExtractSubvector(DL, MVT::v16f16, X, 0);
  SDValue High = DAG.getExtractSubvector(DL, MVT::v16f16, X, 16);
  SDValue ExpLow = DAG.getExtractSubvector(DL, MVT::v16i16, Exp, 0);
  SDValue ExpHigh = DAG.getExtractSubvector(DL, MVT::v16i16, Exp, 16);

  SDValue OpLow = DAG.getNode(ISD::FLDEXP, DL, MVT::v16f16, Low, ExpLow);
  SDValue OpHigh = DAG.getNode(ISD::FLDEXP, DL, MVT::v16f16, High, ExpHigh);
  SDValue ScaledLow = LowerFLDEXP(OpLow, Subtarget, DAG);
  SDValue ScaledHigh = LowerFLDEXP(OpHigh, Subtarget, DAG);
  return DAG.getNode(ISD::CONCAT_VECTORS, DL, MVT::v32f16, ScaledLow,
                       ScaledHigh);

Which after being pointed out by the PR’s reviewer, I found out the splitting/concatenating can be simply reduced to the helper function below.

  return splitVectorOp(Op, DAG, DL);

The code below handles the cases where VLX is not supported, in which the vectors need to be widened to 512-bits.

  SDValue WideX = widenSubVector(X, true, Subtarget, DAG, DL, 512);
  SDValue WideExp = widenSubVector(Exp, true, Subtarget, DAG, DL, 512);
  Exp = DAG.getNode(ISD::SINT_TO_FP, DL, WideExp.getSimpleValueType(), Exp);
  SDValue Scalef =
      DAG.getNode(X86ISD::SCALEF, DL, WideX.getValueType(), WideX, WideExp);
  SDValue Final =
      DAG.getExtractSubvector(DL, X.getSimpleValueType(), Scalef, 0);
  return DAG.getFPExtendOrRound(Final, DL, XTy);

Conclusion

Overall, that was a pretty fun PR. I’m very grateful to my reviewer for his patience. If at any point this blog gave the impression that I got this PR done easily, please take a look at the PR link to see the many iterations of code reviews. It definitely pushed me to write code in a more optimal manner. It was also satisfying trying to find ways to reduce the number of lines I introduce, and is something I’ll focus on as I keep contributing. Looking forward to my next PR.

Resources

https://cdrdv2.intel.com/v1/dl/getContent/671200
- Intel’s manual which contains documentation on the AVX-512 instructions, including VSCALE
https://llvm.org/doxygen/classllvm_1_1SelectionDAG.html
https://www.youtube.com/watch?v=nNQ6AF6i5FI

Nes Emulator

Mon, 09 Jun 2025 19:06:00 -0400

This blog covers my journey of emulating the NES, a popular console from the 1980s made by Nintendo.

Yet Another NES Emulator

Why write an emulator for the NES, a project that can be considered well-trodden at this point? For those pursuing low level programming, an emulation project can serve as an excellent appetizer to the field. It gets you familiarized with CPU architecture and the interactions of different components within a computer. It’s a project big enough that authors have to concern themselves with profiling, and unit testing, concepts that may not be well taught in university. With a sufficiently complex computer to emulate, authors will actually have to think about how to structure their project, else be ridden with spaghetti code that can be a nightmare to debug. And why not the NES, which at this point is well documented and has a lot of support from communities such as r/EmuDev.

The intentions of this blog are to document my process in writing this emulator, as well as write about the flow of execution of the PPU, and the APU. I will not go into the specifics, as the nesdev wiki already provides the necessary technical details of the NES. I hope that my blog can serve as a guide to what each component does on a higher level.

Core Emulation Logic

CPU - 6502

CPU Components

I would like to preface this by saying that when I refer to CPU components, I am talking about the elements of the 6502 relevant to emulation. A good starting point would be to list the registers of the 6502 CPU.

A register is simply a piece of hardware that contains bits of data. For the sake of emulation, registers can be represented as unsigned integers.

PC (Program Counter)

The PC is a 16 bit register that tracks where the program is at in memory. It can be thought of as an index of an array, with the array being memory.

A (Accumulator)

This is an 8 bit register that is typically used to hold the result of operations performed by instructions.

X, Y

These are 8 bit registers, and typically used as operands for operations.

S

This 8 bit register serves as a stack pointer, letting the CPU know where the top (or bottom in the case of the NES) of the stack is at.

P

This is the status register. It’s not really a register, but can be thought of as one. Each bit in this ‘register’ represents a state after an instruction is performed. Decimal mode can be ignored, as this is not a feature on the NES.

Bit	Name	Flag	Description
7	Negative	N	Set if the result of an operation is negative (bit 7 set)
6	Overflow	V	Set if signed overflow occurred
5	Unused	—	Always set to 1 when pushed to the stack, ignored otherwise
4	Break	B	Set when a BRK instruction was executed
3	Decimal	D	Decimal mode (ignored on NES)
2	Interrupt Disable	I	If set, disables IRQ interrupts
1	Zero	Z	Set if the result of an operation is zero
0	Carry	C	Set if the last addition or shift resulted in a carry or borrow

Memory

The CPU also has access to 64KB of memory, with each element of this memory being able to store an 8 bit value. The table below is the CPU’s memory layout. Take a careful look at this map, particularly region $8000 to $FFFF, as this is where the games discussed in this blog will be stored.

Address Range	Size	Purpose
`$0000–$07FF`	`$0800`	2 KB internal RAM
`$0800–$0FFF`	`$0800`	Mirrors of `$0000–$07FF`
`$1000–$17FF`	`$0800`	Mirrors of `$0000–$07FF`
`$1800–$1FFF`	`$0800`	Mirrors of `$0000–$07FF`
`$2000–$2007`	`$0008`	PPU MMIO registers
`$2008–$3FFF`	`$1FF8`	Mirrors of `$2000–$2007` (repeats every 8 bytes)
`$4000–$4017`	`$0018`	NES APU and I/O registers
`$4018–$401F`	`$0008`	APU and I/O functionality (normally disabled — see CPU Test Mode)
`$4020–$FFFF`
`$6000–$7FFF`		Usually cartridge RAM, when present
`$8000–$FFFF`	`$BFE0`	Usually cartridge ROM and mapper registers

The code below is a representation of the information relayed so far as a struct in C. There are some values that weren’t covered yet and they can be ignored for now as they do not pertain to the core cpu functionality.

typedef struct Cpu6502 {

  int cycles;

  uint8_t instr;

  // Accumulator
  uint8_t A;

  // X and Y register
  uint8_t X;
  uint8_t Y;

  // Program Counter
  uint16_t PC;

  // Stack pointer
  // Initialize it to the top
  uint8_t S;

  unsigned char P[8];

  PPU *ppu;

  uint8_t ctrl_latch_state;
  int ctrl_bit_index;

  unsigned char nmi_state;
  unsigned char strobe;
} Cpu6502;

Stack

The stack is a part of the memory used to temporarily hold values assigned by the CPU. It’s used to save values when the CPU needs to change to a different context. Certain instructions, such as the Jump to Subroutine (JSR) instructs the CPU to change the program counter value to another value. The current PC will be saved by pushing its value to the stack.

CPU Instructions

The CPU executes instructions and these instructions can vary in function. The simplest instructions implement are the read/writeand the register operations. The LDA, LDX, LDY, STA, STX, STY are the read & write instructions of the 6502 ISA. The last letter (A, X, or Y) indicates the register that will be used in the instruction. LD retrieves a value from memory, and the location of this value is determined by the Addressing Mode. The retrieved value is stored in the specified register. For example, LDA # gets a value from the immediate address and stored it in the accumulator register. Most of the CPU’s instructions are quite simple functionality-wise so I won’t cover them. This site, as well the NesDev Wiki fully documents the 6502 ISA.

Addressing Mode

Certain instructions need operand(s) to execute. For example, a load instruction needs an address to specify where to load the value from memory.

In the code implementation in this section, you will notice that I increment PC within the addressing mode logic. This is not entirely accurate but it serves my purposes, since I do not increment PC in the previous instruction.

Immediate Addressing

PC++;
address = PC;

Immediate Addressing

The immediate address is the value itself, located at PC + 1. This addressing mode fetches the operand directly following the opcode.

    graph TD
    A["Mnemonic: LDA #$42"]
    B["$A9 $42"] 
    C["A ← $42"]
    B --> C

Zero Page Addressing

PC++;
address = memory[PC];

Zero Page Addressing

Zero page refers to memory addresses from $0000 to $00FF. This region is faster for the CPU to access. The instruction uses a single-byte operand (from PC + 1) as the full address.

    graph TD 
    A["Mnemonic: LDA $42"]
    B["$A5 $42"] 
    C["Address [$42] → $10"]
    D["A ← $10"]

    B --> C --> D

Zero Page,X

PC++;
address = (memory[PC] + X) & 0xFF;

Zero Page,X Addressing

This uses the value stored in memory at the address of (PC + 1) incremented by the value in register X. It wraps around within the zero page using & 0xFF.

    graph TD 
    M["Mnemonic: LDA $42,X"]
    I["$B5 $42"] 
    Xreg["X → $04"] 
    AddrCalc["Effective address = ($42 + $04) & 0xFF = $46"]
    MemRead["Address [$46] → $99"]
    Result["A ← $99"]

    I --> AddrCalc --> MemRead --> Result

Zero Page,Y

PC++;
address = (memory[PC] + Y) & 0xFF;

Zero Page,Y Addressing

Works the same way as Zero Page,X, but adds the Y register instead. Rarely used in 6502.

Absolute Addressing

lsb = memory[++PC];
msb = memory[++PC];
address = (msb << 8) | lsb;

Absolute Addressing

Absolute addressing requires two operands (bytes) from memory to form a full 16-bit address. The low byte (LSB) comes first, then the high byte (MSB). These are combined using bit shifting and bitwise OR.

    graph TD 
    M["Mnemonic: LDA $1234"]
    I["$AD $34 $12"] 
    Addr["Effective address = $1234"]
    MemRead["Address [$1234] → $56"]
    Result["A ← $56"]

    I --> Addr --> MemRead --> Result

Absolute,X

lsb = memory[++PC];
msb = memory[++PC];
address = ((msb << 8) | lsb) + X;

Absolute,Y

lsb = memory[++PC];
msb = memory[++PC];
address = ((msb << 8) | lsb) + Y;

(Indirect,X)

PC++;
base = (memory[PC] + X) & 0xFF;
lo = memory[base];
hi = memory[(base + 1) & 0xFF];
address = (hi << 8) | lo;

(Indirect,X) Addressing

First adds X to the operand (from zero page), then uses the result as a pointer to a 16-bit address. The pointer wraps in zero page.

    graph TD
    M["Mnemonic: LDA ($20,X)"]
    I["$A1 $20"]
    Xreg["X → $04"]
    AddrCalc["$20 + $04 = $24"]
    PtrLow["Address [$24] → $78 (low byte)"]
    PtrHigh["Address [$24 + $1] → $56 (high byte)"]
    EffAddrLow["Effective address low byte: $78"]
    EffAddrHigh["Effective address high byte: $56"]
    EffAddr["Effective address = $5678"]
    MemRead["Address [$5678] → $9A"]
    Result["Load A ← $9A"]

    M --> I --> Xreg --> AddrCalc
    AddrCalc --> PtrLow --> PtrHigh
    PtrLow --> EffAddrLow
    PtrHigh --> EffAddrHigh
    EffAddrLow --> EffAddr
    EffAddrHigh --> EffAddr
    EffAddr --> MemRead --> Result

(Indirect),Y

PC++;
ptr = memory[PC];
lo = memory[ptr];
hi = memory[(ptr + 1) & 0xFF];
address = ((hi << 8) | lo) + Y;

(Indirect),Y Addressing

Reads a pointer from zero page, builds a base address from it, then adds the Y register to get the effective address.

    graph TD
    M["Mnemonic: LDA ($20),Y"]
    I["$B1 $20"]
    PtrLow["Read low byte at Address [$20]: $34"]
    PtrHigh["Read high byte at Address [$20 + 1]: $12"]
    Yreg["Y register → $05"]
    AddrCombine["Combine bytes: $1234"]
    EffAddrCalc["Add Y: $1234 + $05 = $1239"]
    MemRead["Address [$1239] → $BC"]
    Result["A ← $BC"]

    M --> I --> PtrLow --> AddrCombine 
    I --> PtrHigh --> AddrCombine 
    AddrCombine --> EffAddrCalc --> MemRead --> Result
    Yreg --> EffAddrCalc

PPU - The Picture Processing Unit

This component of the NES is what generates the graphics. The PPU was definitely the most confusing and time consuming part of this emulation process. The documentation I was able to find was in-depth on the individual components of the PPU, but I was confused as to what the PPU was doing at a high level. If you’re having a hard time understanding what the PPU is doing, I hope this initial high-level overview helps.

The main goal of the PPU is to draw a frame, pixel by pixel. The visible frame is the actual display, 256 by 240 pixels. This can also be thought of as 32 by 30 tiles, with each tile being 8 by 8 pixels. To draw each pixel, the PPU will have to know what colour each pixel is. The PPU does 4 memory fetches every 8 cycles to get the colour of a pixel. The first memory fetch is from the Nametable using the value in the PPU’s v register. The Nametable contains the tile ID. For example, let’s say the Nametable returns $13 (19 in base 10), this means we have to use the 19th tile from the CHR, which is stored in the Pattern Table. The next two cycles are spent on a memory fetch from the Attribute Table. The Attribute Table determines which Palette out of the four palettes to use for a 4x4 tile region. The next 4 cycles are spent on the Pattern Table. A palette is composed of four possible colours to use. The Pattern Table will determine which of the four colours in a palette to use. All these memory fetches will let us build an address that reveals which colour from the Palette memory to use.

    flowchart TD
    A["PPU Begins Rendering Pixel"] --> V(["v = $2000"])

    V --> B["Fetch Tile ID from Nametable"]
    B --> C(["Tile ID = $13"])

    V --> AT["Fetch Palette Index from Attribute Table"]
    AT --> G(["Palette Index = 3"])

    C --> D["Fetch Pattern from Pattern Table"]
    D --> E(["Tile #19 Pixel Data"])

    E --> F["Combine Pattern Data + Palette Index"]
    G --> F

    F --> H["Fetch Color from Palette Memory"]
    H --> I(["Final Pixel Color = #12"])

This diagram simplifies the flow a bit too much. There are some crucial nuances that the Wiki will cover, and some that I’ll try to cover below.

PPU Memory Mapped & Internal Registers

Memory mapped registers are essentially the channels of communication between the CPU and PPU. These two components have seperate memory space. For these two components to communicate with each other, the PPU has eight registers which are mapped to certain addresses of the CPU in its memory space. For example, the PPU has a register called PPUCTRL. The CPU is writing to this register when it writes to the address $2000 in its memory space. I won’t cover in detail the purpose of each memory mapped register as the NesDev Wiki details this extensively.

    flowchart LR
    A["A → $80"] 
CPU["STA $2000"]
    CPU --> Write["Stores $80 in CPU's memory address $2000"]
    Map["$2000 is Memory-Mapped to PPUCTRL Register"] --> PPUCTRL["PPUCTRL Register ← $80"]
    PPUCTRL --> PPU["PPU Updates Control Flags"]

Frames, Scanlines, & Cycles

A frame is an instance of the screen. Each frame is composed of 262 scanlines, which can be thought of as rows. Earlier, I mentioned the visible frame, which can also be referred to as the picture region. But I also said there were 240 rows. This discrepancy is due to something called the Vertical Blanking Lines (VBlank). The PPU has essentially finished drawing the frame by scanline 239, but there are additional scanlines, (241 to 260) where the PPU is idle. During this period of time, the CPU can commit operations that can alter PPU’s behaviour, such as changing which sprites to appear on the next frame, or change the palette colours. Scanline 0 to scanline 239 are referred to as the Visible Scanlines. Scanline 240 is the Post-render scanline, where the PPU is idle. The VBlank flag isn’t yet so this is not VBlank. At scanline 241, if enabled in PPUCTRL the PPU triggers an NMI, which notifies the CPU that the PPU is in VBlank.

Scanline #	Description
0–239	Visible Scanlines
240	Post-render (idle)
241–260	VBlank (Vertical Blanking)
261	Pre-render scanline

Each scanline requires 341 PPU clock cycles, with each clock cycle producing one pixel. You may notices the additional 86 cycles to draw the 256 pixels per row. Cycle 0 is idle, it does nothing. Cycle 1 to Cycle 256 is where the 4 memory fetches I mentioned above are executed. Cycle 257-320 are pre-fetching the first two tiles of the next scanline. Cycle 337-340 also does memory fetches, but their purpose are unknown and for the sake of emulation can be ignored.

Cycle Range	Purpose / Memory Fetch Pattern
0	Idle (no operation)
1–256	Pixel rendering (4 memory fetches every 8 cycles)
257–320	Prefetch tiles for next scanline
321–336	Pipeline warm-up / dummy fetches
337–340	Unused or unknown-purpose fetches (safe to ignore)

Cycle % 8	Memory Fetch Target	Description
1	Nametable byte	Tile ID for the current tile
3	Attribute table byte	Palette attribute for 4×4 tile region
5	Pattern table low byte	LSB of tile graphics from CHR
7	Pattern table high byte	MSB of tile graphics from CHR

Memory Reads

Nametable

A nametable dictates how the background is laid out. Each nametable is composed of 1024 bytes, each byte storing the tile ID of each of the 30x32 tiles. So 960 bytes (30 * 32) are used to indicate which tile to use from the pattern table, the remaining 64 bytes are used by each nametable’s Attribute Table. The CPU will write to this memory area ($2000 - $2FFF) as instructed by the PRG data, so the CPU (indirectly) directed by the PRG data determines how the background is laid out.

inline uint8_t read_mem(PPU *ppu, uint16_t addr) {
  
  ... 

  else if (addr < 0x3F00) {
    // Nametable region with mirroring
    addr &= 0x2FFF;
    uint16_t offset = addr & 0xFFF;

    if (nametable_mirror_flag) { // vertical
      return ppu_memory[offset % 0x800];
    } else { // horizontal
      if (offset < 0x800) {
        // NT0 or NT1 (2000 or 2400)
        return ppu_memory[(0x2000 | (offset % 0x400))];
      } else {
        if (offset < 0xC00) {
          return ppu_memory[addr];
        } else {
          return ppu_memory[0x2000 | (offset - 0x400)];
        }
      }
    }
  }

  ...

}

The function above is how I fetch the nametable byte using the value in register v. The code handles PPU memory reads in the nametable region ($2000–$2FFF), applying mirroring rules. For vertical mirroring, it mirrors every 2 KB. For horizontal, it redirects specific 1 KB chunks to simulate NT0 and NT1 sharing. The goal is to map the 4 nametable address ranges into the 2 KB of physical VRAM available in the NES.

void background_ppu_render(PPU *ppu) {
  
  ...

  switch (ppu->current_scanline_cycle % 8) {

  // Fetch Nametable byte
  case 1:

    ppu->bg_pipeline.name_table_byte = fetch_name_table_byte(ppu);
    break;

  ...

  }
}

The code above is how the function (fetch_name_table_byte just calls read_mem) is used in the actual background rendering pipeline. The modulo switch statement lets us know which cycle we are on in the 8 cycle pipeline of rendering a tile.

Attribute Table

As mentioned before, the last 64 bytes of each nametable is used as the Attribute table of the corresponding nametable. Each byte controls the palette of a 4x4 tile region of the nametable. Each 2 bits of the byte controls a 2x2 tile region.

Bits	Bit Position	Affected Quadrant	Description
1 0	Bits 0–1	Bottom Right (BR)	Palette index for bottom-right
3 2	Bits 2–3	Bottom Left (BL)	Palette index for bottom-left
5 4	Bits 4–5	Top Right (TR)	Palette index for top-right
7 6	Bits 6–7	Top Left (TL)	Palette index for top-left

void background_ppu_render(PPU *ppu) {
  
  ...

  case 3:
    ppu->bg_pipeline.attribute_byte = fetch_attr_table_byte(ppu);
    // Determines which of the four areas of the attribute byte to use
    // Each attribute byte covers 4x4 tiles → each quadrant is 2x2 tiles
    uint8_t quadrant_x = ((ppu->v & 0x1F) >> 1) & 1; // 0 = left, 1 = right
    uint8_t quadrant_y =
        (((ppu->v >> 5) & 0x1F) >> 1) & 1; // 0 = top,  1 = bottom

    uint8_t shift = (quadrant_y << 1 | quadrant_x) * 2; // 0, 2, 4, or 6
    ppu->bg_pipeline.palette_index =
        (ppu->bg_pipeline.attribute_byte >> shift) & 0x3;
    break;

  ...

}

The code above is the logic I use to extract the relevant part of the attribute byte depending on the register v pointer.

It’s crucial that the logic used here is sound. Previously, I was using if statements to get parse out the attribute byte shown below.

  uint8_t tile_area_horizontal = (ppu->v & 0x00F) % 4;
  uint8_t tile_area_vertical = ((ppu->v & 0x0F0) / 0x20) % 4;

  if (tile_area_vertical == 0 && tile_area_horizontal == 0) {
      // Top left quadrant (bit 1 & 0)
      ppu->palette_index = ppu->attribute_byte & 0x03; :> [!WARNING]
      > 
  } else if (tile_area_vertical == 0 && tile_area_horizontal == 1) {
      // Top right quadrant (bit 3 & 2)
      ppu->palette_index = (ppu->attribute_byte & 0x0C) >> 2;
  } else if (tile_area_vertical == 1 && tile_area_horizontal == 0) {
      // Bottom left quadrant (bit 5 & 4)
      ppu->palette_index = (ppu->attribute_byte & 0x30) >> 4;
  } else if (tile_area_vertical == 1 && tile_area_horizontal == 1) {
      // Bottom right quadrant (bit 7 6 6)
      ppu->palette_index = (ppu->attribute_byte & 0xC0) >> 6;
  }

The code wasn’t computing the tile’s quadrant within the 4-tile attribute area correctly. As a result, the following occurred. There was a bag of other issues with my implementation at the time, but the incorrect colours were mainly due to how I was parsing the attribute byte.

Pattern Table (CHR)

Each tile is consists of 16 bytes, made up of two planes. The first plane controls bit 0 of pixel’s palette index, and the corresponding bit in the second plane controls bit 1. The diagram below will hopefully clear up any confusion. In the example below, PPUCTRL sets the base address of the pattern table to 0x1000, and the nametable byte is 0x3A.

    flowchart TD
    NTB["Nametable Byte: 0x2000:0x3A"]
    PTAddr["Pattern Table Addr: 0x1000 + 0x3A×16 + scanline"]
    PTByte1["Pattern Table Byte Lo: 0x13A0:0x5C"]
    PTByte2["Pattern Table Byte Hi: 0x13A1:0xA3"]

    NTB --> PTAddr --> PTByte1
    PTAddr --> PTByte2

Pixel #	PT Hi Bit	PT Lo Bit	Palette Index (binary)	Palette Index (decimal)
7	1	0	10	2
6	0	1	01	1
5	1	0	10	2
4	0	1	01	1
3	0	1	01	1
2	0	1	01	1
1	1	0	10	2
0	1	0	10	2

void background_ppu_render(PPU *ppu) {

  ...

  case 5:
    ppu->bg_pipeline.pattern_table_lsb =
        read_mem(ppu, (ppu->bg_pipeline.name_table_byte * 16) + fine_y);
    break;

  // Fetch high byte
  case 7:
    ppu->bg_pipeline.pattern_table_msb =
        read_mem(ppu, (ppu->bg_pipeline.name_table_byte * 16) + 8 + fine_y);
    break;

  ...

}

inline uint8_t read_mem(PPU *ppu, uint16_t addr) {

  ...

  if (addr < 0x2000) {
    // Pattern tables
    uint16_t pt_base_addr = 0;

    // if (ppu->drawing_bg_flag) {
    pt_base_addr = (ppu->PPUCTRL & 0x10) ? 0x1000 : 0x0000;
    //} else {
    //  // TODO: Handle 8x16 sprites separately in sprite code
    //  pt_base_addr = (ppu->PPUCTRL & 0x08) ? 0x1000 : 0x0000;
    //}
    return ppu_memory[pt_base_addr | (addr & 0x0FFF)];
  }

  ...

}

For the longest time, my emulator was outputting garbage visuals and it was due to how I was fetching my pattern table data. Currently, my implementation is fine (with the limited testing I have done so far, so this statement is subject to change). But initially, I wrote the code as below.

           
  ...

  case 5:

    if (drawing_bg_flag) {
        ppu->pattern_table_lsb = read_mem(ppu, (ppu->name_table_byte * 16) + ppu->scanline);
        ppu->pattern_table_msb = read_mem(ppu, (ppu->name_table_byte * 16) + ppu->scanline + 8);

    } else {
        // byte 1 is tile number
        ppu->pattern_table_lsb = read_mem(ppu, (oam_buffer_latches[sprite_index + 1] * 16) + ppu->scanline);
        ppu->pattern_table_msb = read_mem(ppu, (oam_buffer_latches[sprite_index + 1] * 16) + ppu->scanline + 8);
    }

    ...
    
  break;

The crucial error is in the address I’m reading from.

ppu->pattern_table_lsb = read_mem(ppu, (ppu->name_table_byte * 16) + ppu->scanline);
ppu->pattern_table_msb = read_mem(ppu, (ppu->name_table_byte * 16) + ppu->scanline + 8);

I was using the incorrect address here! My goal was to increment the name_table_byte by the current row being rendered in a tile, which can range in value from 0 to 8. However, I was increasing by the total number of scanlines! This outputted some interesting visuals when I tried to run the Donkey Kong rom.

Palette

The PPU memory (0x3F00 - 0x3F1F) contains four palettes, with each palette containing four colours. Memory address $3F00 to $3F0F contains the colours that background will use, while memory address $3F10 to $3F1F contains the colours used by sprites. The table below illustrates this.

Address Range	Purpose	Notes
$3F00	Universal background color	Shared by all background palettes
$3F01–$3F03	Background palette 0	3 colors (color 0 is $3F00)
$3F04–$3F06	Background palette 1
$3F07–$3F09	Background palette 2
$3F0A–$3F0C	Background palette 3
$3F0D–$3F0F	Sprite palette 0	$3F0F may mirror $3F10
$3F10–$3F12	Sprite palette 1	$3F10 often mirrors $3F00
$3F13–$3F15	Sprite palette 2
$3F16–$3F18	Sprite palette 3
$3F19–$3F1F	Mirrors	Mirrors previous palette entries

As I mentioned before, each memory fetch plays a part in computing the address used to fetch a pixel’s colour.

Bit	Meaning
3	Sprite (1) or Background (0)
2-1	Attribute bits (palette selection: 0–3)
0	Pixel value from pattern table (0 or 1)

Hopefully everything comes together with this. It took me a while of reading through the NesDev Wiki forums and documentation to piece it together myself.

void background_ppu_render(PPU *ppu) {

    ...

    case 0:

    ...

    unsigned char is_pre_fetch =
        ppu->current_scanline_cycle >= 321 && ppu->current_scanline_cycle <= 336
            ? 1
            : 0;

    int row = is_pre_fetch ? ppu->scanline + 1 : ppu->scanline;
    int column_base = is_pre_fetch ? ppu->current_scanline_cycle - 328
                                   : ppu->current_scanline_cycle + 8;

    for (int i = 0; i < TILE_SIZE; i++) {
      uint8_t bit = 7 - i;
      uint8_t bg_lo = (ppu->bg_pipeline.pattern_table_lsb >> bit) & 1;
      uint8_t bg_hi = (ppu->bg_pipeline.pattern_table_msb >> bit) & 1;

      ppu->bg_pipeline.tile_pixel_value[i] = (bg_hi << 1) | bg_lo;

      uint8_t palette_index = ppu->bg_pipeline.palette_index;

      uint16_t pal_addr =
          0x3F00 | (palette_index << 2) | ppu->bg_pipeline.tile_pixel_value[i];

      uint8_t palette_data = read_mem(ppu, pal_addr);
      uint8_t alpha = (ppu->bg_pipeline.tile_pixel_value[i] == 0) ? 0x00 : 0xFF;

      uint8_t *color = &ppu_palette[palette_data * 3];
      uint8_t r = color[0], g = color[1], b = color[2];

      int column = column_base + i;

      if (column >= 256 || row >= 240)
        break;

      ppu->frame_buffer[row][column] = (r << 24) | (g << 16) | (b << 8) | alpha;
    }
    break;

    ...

}

    sequenceDiagram
    participant PPU
    participant PPU_Memory
    participant FrameBuffer
    participant Frontend 

    loop 8-cycle fetch
        PPU->>PPU_Memory: Fetch nametable byte
        PPU->>PPU_Memory: Fetch attribute byte
        PPU->>PPU_Memory: Fetch pattern LSB
        PPU->>PPU_Memory: Fetch pattern MSB
        PPU->>FrameBuffer: Fill up framebuffer with 8 pixels
    end

    FrameBuffer->>Frontend: Present frame buffer to render

The Nes outputs a pixel cycle by cycle. I’m doing it a little differently. At every 8 cycle from cycle 1 - 256 (as well as the prefetch cycle 321 - 336), I use the information I have fetched the past 8 cycles rendering the past 8 pixels. Once I have a frame’s worth of pixels filled up in the buffer, I pass it off to Frontend, which is an abstraction I made that does all the user interactions, including rendering the graphics.

OAM (Sprites)

The OAM is another internal memory of the PPU that contains up to 64 sprites, with each sprite taking up 4 bytes. The table below can summarize what each of the 4 bytes determine.

Byte	Description	Details
0	Y Position	Y coordinate of top of sprite minus 1 (due to 1-scanline delay). Values #$EF–#$FF hide the sprite offscreen.
1	Tile Index Number	Tile number in pattern table. For 8x16 sprites, bit 0 selects tile bank ($0000 or $1000).
		Bit 0: Bank ($0000 or $1000); Bits 1-7: Tile number top of sprite (0–254).
2	Attributes	Bit 0-1: Palette (4–7) of sprite
		Bit 2-4: Unimplemented (always read 0)
		Bit 5: Priority (0=front of BG, 1=behind BG)
		Bit 6: Flip sprite horizontally
		Bit 7: Flip sprite vertically
3	X Position	X coordinate of left side of sprite

From this table, it can be discerned that the same rendering process done for the background pipeline cannot be used for the sprite. At least that was my conclusion. Instead I made a different function to render and detect the sprites. To be honest, I’m still not fully confident in my sprite rendering implementation. It works with the games I’ve tried so far but I suggest you take this section with a grain of salt, as its most likely subject to change when I realize I did something wrong. Still, I believe my understanding should be able to help somewhat if you’re absolutely lost.

To start with, each scanline from cycle 65-256 skims through the OAM memory to fill up a buffer with any sprite that will appear on the next scanline. At HBlank, after cycle 256, this buffer is transferred to the appropriate shift and rendering registers to be rendered on the next scanline. To represent this, I employ two arrays. One will fill up the sprites of the next scanline, and at HBlank, will transfer its content to the second array so that it can be used to render the sprites on the next scanline.

    flowchart TD
    Start([Start: n=0, m=0])

    ReadY["Read sprite Y-coordinate OAM[n][0]"]
    CheckInRange{"Y in range?"}
    CopySprite["Copy sprite data to secondary OAM if < 8 sprites found"]
    WriteIgnore["Ignore write if secondary OAM full"]
    IncrementN["Increment n"]
    CheckNZero{"All 64 sprites checked (n=0)?"}
    CheckSpriteCount{"< 8 sprites found?"}
    EvaluateSprites["Evaluate sprites for overflow"]
    SetOverflow["Set sprite overflow flag if > 8 sprites"]
    IncrementM["Increment m (hardware bug)"]
    ContinueEval["Continue evaluation"]
    CopyFail["Fail to copy sprite to secondary OAM"]
    IncrementNUntilHBLANK["Increment n until HBLANK"]
    End([End at HBLANK])

    Start --> ReadY
    ReadY --> CheckInRange

    CheckInRange -- Yes --> CopySprite
    CopySprite --> IncrementN
    CheckInRange -- No --> IncrementN

    IncrementN --> CheckNZero
    CheckNZero -- No --> CheckSpriteCount
    CheckNZero -- Yes --> CopyFail

    CheckSpriteCount -- Yes --> ReadY
    CheckSpriteCount -- No --> EvaluateSprites

    EvaluateSprites --> SetOverflow
    SetOverflow --> IncrementM
    IncrementM --> ContinueEval
    ContinueEval --> EvaluateSprites

    CopyFail --> IncrementNUntilHBLANK --> End

If this confuses you, don’t worry cause it confused me too. Just take your time reading through the wiki.

This is the code I was able to come up with my current understanding.

void sprite_detect(PPU *ppu) {
  // Reset secondary OAM memory at cycle 1
  if (ppu->current_scanline_cycle == 1) {
    memset(oam_memory_secondary, 0xFF, OAM_SECONDARY_SIZE);
  }

  if (ppu->current_scanline_cycle == 336) {
    ppu->copy_sprite_flag = 0;
    ppu->index_of_sprite = 0;
    ppu->sprite_evaluation_index = 0;
    ppu->oam_memory_top = 0;
  }

  // Bit 5 of PPUCTRL determines sprite size
  int sprite_height = ppu->PPUCTRL & 0x20 ? 16 : 8;

  if (ppu->current_scanline_cycle >= 65 && ppu->current_scanline_cycle <= 256) {
    // int index = ppu->sprite_evaluation_index * 4 + ppu->index_of_sprite;
    int index = ppu->sprite_evaluation_index * 4;
    if (index > OAM_SIZE - 1)
      return;
    // Odd cycle; read OAM
    if (ppu->current_scanline_cycle % 2 == 1) {
      if (ppu->scanline >= oam_memory[index] &&
          ppu->scanline < oam_memory[index] + sprite_height) {
        ppu->copy_sprite_flag = 1;
      }
    } else {
      if (ppu->copy_sprite_flag && ppu->oam_memory_top <= 31) {
        oam_memory_secondary[ppu->oam_memory_top++] = oam_memory[index];
        oam_memory_secondary[ppu->oam_memory_top++] = oam_memory[index + 1];
        oam_memory_secondary[ppu->oam_memory_top++] = oam_memory[index + 2];
        oam_memory_secondary[ppu->oam_memory_top++] = oam_memory[index + 3];
      }
      if (ppu->sprite_evaluation_index == 64) {
        ppu->sprite_evaluation_index = -1;
        ppu->PPUSTATUS |= 0x20;
      }
      ppu->sprite_evaluation_index++;
      ppu->copy_sprite_flag = 0;
    }
  }
}



void ppu_exec_visible_scanline(PPU *ppu) {

  ...

  oam_buffer_latches[(ppu->current_scanline_cycle - 257) % 32] =
    oam_memory_secondary[(ppu->current_scanline_cycle - 257) % 32];
  
  ...
}

The actual rendering code is basically identical to the background rendering. It just uses data from the oam_buffer_latches instead ofppu_memory.

Loop v

v is the current VRAM address used by the PPU to fetch the nametable, and attribute data. It’s a 15-bit register that changes during the rendering process, and each bit of this register has a significance to which tile to retrieve.

Bit(s)	Name	Description
14–12	fine Y	Row offset within a tile (0–7)
11	nametable Y	Selects vertical nametable (0 or 1)
10	nametable X	Selects horizontal nametable (0 or 1)
9–5	coarse Y	Tile row index on the screen (0–29)
4–0	coarse X	Tile column index on the screen (0–31)

–

To get to this point took me quite a long time, mostly a lot of painful debugging. But after getting some coherent gameplay, it was all worth it.

APU - Audio Processing Unit

Now it’s time to add audio. I found this part to be not too hard compared to the PPU, but the theory behind it can be quite confusing. How does a digital signal get converted to audio? Some basic understanding of digital signal processing will help, but basically sound is just a vibration in the air. The APU generates samples of this vibration as a digital representation. My job is to simply collect these samples and use an existing library, SDL2 in my case, to generate the actual audio. The APU is comprised of five channels, 2 pulse channels, 1 triangle channel, 1 noise channel, and a delta modulation channel. The output of each of these channels is sent to a mixer. Currently, I’m using a simple averaging mixer, but I do plan on exploring other formulas in the future.

uint8_t apu_output(APU *apu) {
  uint8_t p1 = apu_pulse_output(apu->pulse1);
  uint8_t p2 = apu_pulse_output(apu->pulse2);
  uint8_t tri = apu_triangle_output(apu->triangle);
  uint8_t noise = apu_noise_output(apu->noise);
  // At this point, I still have not implemented the Delta modulation channel.
  
  return (p1 + p2 + tri + noise) / 3;
}

Frame Counter

The frame counter basically acts like a clock generator. Similar to how everything acts on the PPU cycle in the PPU, certain components act on the clock signal generated by the frame counter. The frame counter itself derives its generated clock signals by the APU cycle. For example, the frame counter will generate a clock signal at 3728 APU cycles. This would clock the Envelopes and Triangle's linear counter.

Length Counter, Envelope, Sweep

Before I cover the actual channels, I’ll start off with the components they’re comprised. Namely, the length counter, envelope generator and the sweep unit.

The length counter acts like a timer. Once it counts down to zero, the sound will stop. The CPU writes a 5 bit values to one of the APU’s memory mapped register, and this value serves as an index to a duration. For example, lets say the value is $1F. The corresponding duration would be 30, which is clocked by the Frame Counter.

    flowchart LR 
    CPU["CPU writes 5-bit value (e.g. $1F) to APU register"] --> IDX["Value used as index into length table"]
    IDX --> DUR["Length counter initialized (e.g. 30)"]
    DUR --> FC["Frame Counter ticks at 240Hz"]
    FC --> DEC["Length counter decrements each tick"]
    DEC --> CHECK{"Length counter == 0?"}
    CHECK -- Yes --> STOP["Sound channel muted"]
    CHECK -- No --> DEC

The Envelope controls the volume output. Depending on certain flags, the volume can either decay with an optional looping to repeat the decay, or it can be a constant volume.

The Sweep unit can adjust a channel’s period up or down. Shorter period creates a higher pitch, while longer period creates a lower pitch.

Pulse Channels

This is a simple square wave generator with an adjustable duty cycle, volume and sweep. A period is the length of the smallest unrepeatable values in a signal. Duty cycle determines during the period when a value is 0, or when the signal is 1. Volume can either be constant, or have a decay. It is composed of a length counter, envelope, and a sweep unit.

I have included a little clip to show how important it is to nail the logic of each component. Have a listen!

Your browser does not support the video tag.

It took me a long time to pinpoint this issue, as it was due to just a single line.

  if (envelope->constant_volume_flag) {
    envelope->volume = envelope->divider->P;
  } else {
    envelope->volume = envelope->divider->counter;
  }

If the constant volume flag was not set, I was resetting the volume to the value of the divider. This is why each note sounds choppy, as if the volume restarts! The corrected code is as shown below.

  if (envelope->constant_volume_flag) {
    envelope->volume = envelope->divider->P;
  } else {
    envelope->volume = envelope->envelope_decay_level_counter;
  }

This results in the following.

Your browser does not support the video tag.

Triangle Channel

The Triangle channel is composed of a timer, a length counter, a linear counter, and some flags. A linear counter is basically a length counter, but its value is directly used, unlike a length counter which uses an index to get its duration.

Unlike the pulse, the triangle channel has no envelope. Instead, when the timer is clocked, it traverses through a 32 step sequence of values to output to the mixer.

15, 14, 13, 12, 11, 10,  9,  8,  7,  6,  5,  4,  3,  2,  1,  0
 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15

Noise

The Noise channel is composed of an envelope generator, a timer, a Linear Feedback Shift Register, and a length counter. When clocked by the timer, a 1 bit feedback value is calculated, the shift register gets shifted to the right by 1 bit, and the leftmost bit is set to the feedback. The mixer will get the value generated by the envelope if bit 0 if the shift register is 1. It will get zero otherwise.

    flowchart TD
    R["Shift Register (R): 100000000000001"]

    Timer["Timer clocks shift register"] --> Feedback["XOR bit 0 (1) and bit 6 (0) equals 1"]

    Feedback --> ShiftAfter["After shift right by 1 bit: 010000000000000"]

    ShiftAfter --> SetBit14["Set bit 14 = feedback (1)"]

    SetBit14 --> Result["New register value: 1100000000000001"]

    R

DMC

To be implemented.

The Frontend - SDL2

Frontend is just the name of my abstraction for SDL2. This is the library that handles all the rendering, control inputs, and audio. It’s a very popular library with a lot of examples, so it shouldn’t be too difficult getting something up and running using this library.

Visuals

I employ a flag to let the renderer know if the buffer is ready. When scanline 260 is reached, the PPU will set this flag.

void ppu_execute_cycle(PPU *ppu) {

  ...

    if (ppu->scanline > 260) {
      ppu->scanline = -1;
      ppu->update_graphics = 1;
    }

  ...

}

Once the flag is set, the Frontend library will update SDL2 with the PPU’s frame buffer and reset the flag.

while (1) {

  ...

    if (ppu_graphics_flag_set(&ppu)) {
      ppu_graphics_flag_reset(&ppu);

      Frontend_DrawFrame(&frontend, ppu.frame_buffer);
      Frontend_SetFrameTickStart(&frontend);
    }

  ...

}

Controls

The CPU sets a flag, strobe, to indicate when its ready to poll the user inputs. When the user presses a button on the NES controller, the corresponding bit on a register is set, which is fed to the CPU. This register essentially holds the state of the controller per frame.

    if (cpu_strobe_set(&cpu)) {
      if (Frontend_HandleInput(&frontend) != 0)
        break;
      // cpu.ctrl_latch_state = frontend.controller;
      Frontend_SetController(cpu_ctrl_latch_state(&cpu));
    }

The following code shows what keys I use for control.

const static int keyboard[8] = {
    SDL_SCANCODE_Z,  SDL_SCANCODE_X,    SDL_SCANCODE_C,    SDL_SCANCODE_V,
    SDL_SCANCODE_UP, SDL_SCANCODE_DOWN, SDL_SCANCODE_LEFT, SDL_SCANCODE_RIGHT};

int Frontend_HandleInput(Frontend *frontend) {
  SDL_PumpEvents();
  const Uint8 *keystate = SDL_GetKeyboardState(NULL);
  frontend->controller = 0;

  if (keystate[SDL_SCANCODE_Q])
    return 1;
  if (keystate[keyboard[0]])
    frontend->controller |= NES_A;
  if (keystate[keyboard[1]])
    frontend->controller |= NES_B;
  if (keystate[keyboard[2]])
    frontend->controller |= NES_SELECT;
  if (keystate[keyboard[3]])
    frontend->controller |= NES_START;
  if (keystate[keyboard[4]])
    frontend->controller |= NES_UP;
  if (keystate[keyboard[5]])
    frontend->controller |= NES_DOWN;
  if (keystate[keyboard[6]])
    frontend->controller |= NES_LEFT;
  if (keystate[keyboard[7]])
    frontend->controller |= NES_RIGHT;
  return 0;
}

Audio

The standard frequency used for sampling is 44.1 kHz. With that in mind, we need to collect enough samples to generate audio at 44.1 kHz. However, the CPU frequency is 1.79 MHz. This would generate too many samples, at a frequency that would be inaudible to human ears. As such, I had to downsample this high-frequency signal to the standard 44.1 kHz audio rates, which will still preserve the shape of the NES sound waves while remaining audible. The calculation below illustrates how many samples I would need to get per frame.

\[ \begin{aligned} \text{Audio Sample Rate} &= 44{,}100 \, \text{samples/sec} \\ \text{Video Frame Rate (NTSC)} &= 60 \, \text{frames/sec} \\ \\ \text{Samples per Frame} &= \frac{44{,}100}{60} = 735 \, \text{samples/frame} \end{aligned} \]

The calculation below illustrates how many times I’d have to downsample the NES APU signal to get the audio sample rate of 44.1 kHz. What this means is that I have to call my apu mixer function once approximately every 40 CPU cycles.

\[ \begin{aligned} \text{CPU Clock Frequency} &= 1{,}789{,}773 \text{ Hz} \\ \text{Audio Sample Rate} &= 44{,}100 \text{ Hz} \\ \\ \text{Downsampling Factor} &= \frac{\text{CPU Clock Frequency}}{\text{Audio Sample Rate}} \\ &= \frac{1{,}789{,}773}{44{,}100} \\ &\approx 40.57 \end{aligned} \]

My implementation of the calculation presented above is shown in the code below. I still call the apu mixer output (apu_output) every CPU cycle, but I average it out over every ~40 CPU cycles. I could simply just disregard the other samples and call it every 40 cycles, but the sound quality improves drastically with this simple technique. The audio_buffer_add is a function in the Frontend abstraction that adds the samples to a ring buffer.

    for (int i = 0; i < cpu.cycles; i++) {
      apu_execute(&apu);

      sample_accumulator += apu_output(&apu);
      apu_accumulator += 1.0;

      if (apu_accumulator >= apu_ticks_per_sample) {
        apu_accumulator -= apu_ticks_per_sample;

        int16_t sample =
            ((int)(sample_accumulator / apu_ticks_per_sample) - 8) * 4096;

        audio_buffer_add(sample);

        sample_accumulator = 0.0;
      }
    }

The code below shows how SDL2 initializes the audio. It sets a frequency of 44100 Hz, expects 735 samples per buffer, repeatedly calls audio_callback (the function the parses through the samples in the ring buffer), and uses signed 16 bit.

SDL_AudioSpec spec = {
      .freq = 44100,
      .format = AUDIO_S16SYS,
      .channels = 1,
      .samples = 735,
      .callback = audio_callback,
      .userdata = NULL,
  };

...

void audio_callback(void *userdata, Uint8 *stream, int len) {
  int16_t *out = (int16_t *)stream;
  int samples = len / sizeof(int16_t);

  for (int i = 0; i < samples; ++i) {
    if (read_pos != write_pos) {
      out[i] = audio_buffer[read_pos];
      read_pos = (read_pos + 1) % AUDIO_BUFFER_SIZE;
    } else {
      out[i] = 0;
    }
  }

Wrap up

That’s about it. I still have some stuff to do, like finishing up the DMC, trying out different apu mixer formulas, implementing other NROM mappers. But as to the core functionality, I’m basically done. Of course, I still want to test other NROM0 ROMs, and I’m sure I’ll find something wrong about my code down the line. Overall, this was a pretty fun project. If you want to try something that will get you familiarized with how a computer works, building any emulator will definitely do it.

Rust-based Quadcopter Part 1

Sat, 08 Jun 2024 22:59:08 +0000

This post covers the details of how I made a flight controller for a quadcopter using Rust and an STM32.

Introduction

This project began as a desire to familiarize myself with embedded development. I wanted to challenge myself by using technology I was not familiar with or had no prior knowledge of. This led me to the idea of making a Rust-based quadcopter. I will walk through the different components and software I used to realize this project.

Components

To simplify things on my end, I decided to buy a drone kit that came with the following items:

Chassis
4 30A Electronic Speed Controllers
4 BLDC Motors
4 Propellers
Screws & Bolts

Other components needed:

STM32F411CEU6
FlySky Transmitter & Receiver
LiPO Battery
IMU (LSM6DSOX)

Rust

Rust is a systems programming language with an emphasis on performance and stability. Writing a quadcopter flight controller in C is cool, but it’s been done before. I wanted to walk the less-trodden path and take this as an opportunity to learn a new language as well.

It’s also helpful that Rust has extensive community support for microcontrollers. There are HALs for many microcontrollers provided by STM32. In fact there is even a crate, svd2rust, that allows users to transform SVD files into crates that expose APIs to access the peripherals of the microcontrollers.

I am using the STM32F411CEU6 for my flight controller, so I will be using the stm32f4xx-hal.

Real-Time Interrupt-Driven Concurrency

RTIC is a Rust-based framework that allows real-time interrupt-driven concurrency for ARM Cortex-M microcontrollers. It is similar to an RTOS in that they both perform scheduling. Where they differ is that RTIC uses the hardware on the Cortex-M MCUs to perform the scheduling instead of a scheduling kernel.

IMU & Kalman Filter

IMU, short for inertial measurement unit, measures the angular rate and acceleration of our quadcopter’s roll, pitch, and yaw. These measurements can be used to determine the attitude (roll, pitch, and yaw) of my flight controller.

The IMU I will be using, the LSM6DSOX, consists of a gyroscope and accelerometer. The gyroscope allows for the measurement of the angular rates of the IMU’s roll, pitch, and yaw axes. The accelerometer allows for the measurement of the acceleration of the IMU’s roll, pitch, and yaw axes.

This IMU also supports I2C and SPI communication protocols. These protocols allow for communication with other devices. In my case, I need to send the reported measurements from my IMU to the STM32. While the list of I2C and SPI drivers written in Rust may be extensive, there was not one yet for the LSM6DSOX as of the date of this project, so I wrote my own I2C driver.

I2C Driver

Before writing an I2C driver in Rust, I first took a look at how other drivers were created in Rust. This is a pretty good list to look at. It even contains a few drivers for other IMUs.

Most of them were built on top of the embedded-hal crate. This crate serves as a foundation for building platform-agnostic drivers, allowing a driver to work on a microcontroller regardless of its platform.

Before I go further into details about my I2C driver, a basic understanding of the I2C protocol might be helpful. The I2C protocol uses two wires for serial communication between a Master and a Slave. These wires are labeled the serial data line (SDA) and the serial clock line (SCL). The first line is for data transfer and the second line is for communication. When attempting to read data through the SDA bus, it is first necessary for the Master (or the STM32 in my case) to write the address of the slave onto the bus it wants to read from. This is necessary so that any slave can respond if they have the corresponding address.

This was a very basic overview of the I2C protocol, and a lot of the underlying mechanisms of how it works were not covered, mainly because the embedded-hal crate does a great job at abstracting away the underlying details.

With that basic understanding of the protocol, my driver will make more sense. In order to get the gyroscope and accelerometer values, I just have to write the address of my IMU on the I2C bus, and then read from the address of the register that stores the desired data. To find the IMU slave address, and the address of the registers where the data is stored, I had to read the LSM6DSOX datasheet. I have created two files, gyro.rs and accel.rs, to contain the registers necessary to read their values.

#[allow(non_camel_case_types)]
#[allow(dead_code)]
#[derive(Copy, Clone)]
pub enum Register {
    // Gyro Configuration Register
    CTRL2_G = 0x11,

    // Angular rate pitch axis (X) output
    OUTX_L_G = 0x22,
    OUTX_H_G = 0x23,

    // Angular rate roll axis (Y) output
    OUTY_L_G = 0x24,
    OUTY_H_G = 0x25,

    // Angular rate yaw axis (Z) output
    OUTZ_L_G = 0x26,
    OUTZ_H_G = 0x27,
}

impl Register {
    pub fn addr(&self) -> u8 {
        *self as u8
    }
}

#[allow(non_camel_case_types)]
#[allow(dead_code)]
#[derive(Copy, Clone)]
pub enum Register {
    // ACCELEROMETER CONFIG REG
    CTRL1_XL = 0x10,

    // Accel rate X axis (X) output
    OUTX_L_A = 0x28,
    OUTX_H_A = 0x29,

    // Accel rate Y axis (Y) output
    OUTY_L_A = 0x2A,
    OUTY_H_A = 0x2B,

    // Accel rate Z axis (Z) output
    OUTZ_L_A = 0x2C,
    OUTZ_H_A = 0x2D,
}

impl Register {
    pub fn addr(&self) -> u8 {
        *self as u8
    }
}

I then defined the class/instance of my i2c driver with the paramter I2C. Users will be required to define an I2C object to use this library.

// lsm6dsox driver
pub struct Lsm6dsox<I2C> {
    i2c: I2C,
}
{% endcodeblock %}


{% codeblock lang:rust %}
impl<I2C, E> Lsm6dsox<I2C> 
    where 
        I2C: WriteRead<Error = E> + Write<Error = E>,
    {       
    pub fn new(i2c: I2C) -> Result<Self, E> {
        let lsm6dsox = Lsm6dsox { i2c };
        Ok(lsm6dsox)
    }

    ...
}

The code above serves to implement a struct Lsm6dsox with a parameter I2C such that I2C has the methods WriteRead and Write.

pub fn configure_accel(&self, i2c: &mut I2C) -> Result<(), E>{
    // ODR_XL[7:4] = 0100; sets accelerometer to work at 200 Hz
    // FS[3:2] = 10; sets accelerometer full-scale selection to 4g
    // LPF2_XL_EN[1] = 1; output from first stage digital filtering
    // 0100 10 1 0
    let configuration: u8 = 0x48;
    match i2c.write(SLAVE_ADDRESS, &[CTRL1_XL, configuration]) {
        Ok(_) => Ok(()),
        Err(e) => Err(e),
    }
}

In the function above, I configure the accelerometer in my IMU. Referring to the datasheet, I observe that each bit in the configuration register influences aspects such as the frequency of readings. I utilize the write method to transmit the 8 bit value configuration to the CTRL1_XL register of the device, specifying the slave address as SLAVE_ADDRESS. The process of configuring the gyroscope mirrors that of the accelerometer, albeit with different configuration values and register addresses.

// Read Accelerometer Data
pub fn read_accel(&mut self) -> Result<[f32; 3], E> {
    
    let mut accel_data: [f32; 3] = [0.0, 0.0, 0.0];
    let mut buffer: GenericArray<u8, U6> = unsafe { MaybeUninit::uninit().assume_init() };

    { 
        let buffer: &mut [u8] = &mut buffer;
        self.i2c.write_read(SLAVE_ADDRESS, &[accel::Register::OUTX_L_A.addr()], buffer)?;
        accel_data[0] = ((buffer[1] as i16) << 8 | (buffer[0] as i16)) as f32 * 4.0 / 32768.0;
        accel_data[1] = ((buffer[3] as i16) << 8 | (buffer[2] as i16)) as f32 * 4.0 / 32768.0;
        accel_data[2] = ((buffer[5] as i16) << 8 | (buffer[4] as i16)) as f32 * 4.0 / 32768.0;
    }
    Ok(accel_data)
}

In the function above, I read accelerometer data. Each axis’s data is 16 bits, spread across 2 8-bit registers. To get each axis’s value, I read from 6 consecutive registers (3 axis multiplied by 2 register per axis). The write_read method writes SLAVE_ADDRESS to the I2C bus and reads enough bytes from the device with the address of SLAVE_ADDRESS to fill buffer in one go. This reads all accelerometer registers if buffer has 6 bytes. The IMU is configured by default to automatically increment a register address during a multiple byte access with a serial interface.

Accelerometer data for each axis is 16-bit. The IMU stores the 8 most significant bits in one register and the 8 least significant bits in another. For the accelerometer, OUTX_H_A is the MSB for the X-axis, and OUTX_L_A is the LSB. I left shift buffer[1], buffer[3], and buffer[5] by 8 bits, and bitwise OR these values with their corresponding LSBs to reconstruct each axis’s value.

Deriving the Attitude

So I’m able to get the angular rates and acceleration of each axis using my IMU. But how can this be used to get the angles of roll, pitch, and yaw axis? I’ll discuss two ways of obtaining the attitude from the gyrscope and accelerometer.

Angular rate is just the rate at which the angle changes in a moment of time. By integrating this value, we can get the the angle of each axis. Integration in the context of a microcontroller would have to be discretized. This simplifies the calculations a lot. To get the angle of an axis, you would do a summation of the corresponding angular rate multiplied by the delta time. In our case, the delta time would be the amount of time elapsed between the previous and current loop.

let mono_timer = MonoTimer::new(cp.DWT, cp.DCB, &clocks);
let start_time = mono_timer.now();

loop {
    let time = start_time.elapsed();
    let delta_time = time - prev_time;
    let delta_sec = delta_time as f32/ clocks.sysclk().raw() as f32 * 2.0;

    gyro_data = imu.read_gyro().unwrap();

    x_gyro += delta_sec * gyro_data[0]; // Angle of flight controller in the x-axis
    y_gyro += delta_sec * gyro_data[1]; // Angle of the flight controller in the y-axis
    z_gyro += delta_sec * gyro_data[2];

    prev_time = time;
}

While this method works in the short term, you will notice that the angles derived from the gyrscope drastically loses any accuracy as time progresses. This is due to the fact the there is an uncertainty in the readings of the angular rate, and as we integrate these uncertainties, the error will accumulate.

The second method is using the accelerometer. As previously mentioned the accelerometer measures the acceleration of each of its axis (X, Y, and Z). At rest, the Z axis will report a value of around ~ 1 (measured in g’s). If the IMU is in a different orientation, each of its axis will experience a different portion of gravity. Plugging these values into trigonometric equations can give us the orientation of the IMU in terms of degrees of each of its axis except for the z-axis. The z-axis cannot be calculated as this is inherent limitation to an accelerometer.

accel_data = imu.read_accel(&mut i2c).unwrap();

y_accel = atanf(accel_data[0] / sqrtf(accel_data[1] * accel_data[1] + accel_data[2] * accel_data[2]) ) * 180.0 / PI;
x_accel = atanf(accel_data[1] / sqrtf(accel_data[0] * accel_data[0] + accel_data[2] * accel_data[2]) ) * 180.0 / PI;

While the results from an accelerometer are mostly accurate, in moments of high vibrations the values are basically useless as there is too much noise.

So we have a scenario in which the gyrscope’s data is useful in the short-term, but useless in the long-term and the vice-versa for the accelerometer’s data. A Kalman Filter is perfect for this scenario.

What is a Kalman Filter?

The Kalman filter is an algorithm used for estimating the state of a system in the presence of noisy measurements. It works by iteratively predicting the next state based on a dynamic model and comparing it with actual measurements. The filter optimally combines predictions and measurements, dynamically adjusting their contributions based on their uncertainties. This allows the Kalman filter to provide a more accurate and stable estimate of the system’s true state, making it widely used in applications like sensor fusion for robotics, navigation, and control systems.

In my case, I can use it to fuse the data from the accelerometer and gyroscope that can minimize their respective downfalls to produce a more accurate result. I found this blog by TKJ Electornics to be very helpful in understanding how a Kalman Filter can be applied to a 6-axis IMU.

ESC

The electronic speed controller is a simple piece of circuit that controls the speed of the motors. There are a few methods to communicate with the ESC from my microcontroller to dictate the speed of the controllers. I will be using the simplest method, PWM signals.

What is PWM?

Pulse-width-modulation is essentially rectangular wave signal with a duty cycle and period. A duty signal is the percentage of the signal that is active. A period is the time between each succesive occurence.

For my ESC, I can dictate the desired speed of my motors by sending a 50Hz PWM signal. When the active portion is 1ms, the ESC interprets that as zero throttle. When the active portion is 2ms, the ESC interprets that as full throttle. Any signal with an active portion between 1ms to 2ms will set the speed proportionately. For example, a PWM signal with an active portion of 1.5ms will set the throttle to half the max speed.

For my ESC to work, it needs to initially register a zero throttle signal before it can receive any other signal

PWM Implementation

I first needed to setup the timer channels. Each timer in the STM32 has two channels assosiated to itself. Thus, each timer can generate two differnt PWM signals. For your microcontroller, you must check the pinout sheet to see which pins to use.

let gpioa = dp.GPIOA.split();
let channels = (Channel1::new(gpioa.pa8), Channel2::new(gpioa.pa9));
{% endcodeblock %}

In the codeblock below, I am setting up the channel as a PWM singal with a frequency of 50 Hz. 

{% codeblock lang:rust %}
let pwm = dp.TIM1.pwm_hz(channels, 50.Hz(), &clocks).split();
let (mut ch1, _ch2) = pwm;
let max_duty = ch1.get_max_duty() as u32; 
ch1.enable();

In the stm32f4xx-hal, the set_duty method requires a value rather than a percentage. To send a zero throttle signal, I need to determine the duty cycle with an active portion of 1ms.

Given that a 50 Hz signal has a period of 20ms, the duty cycle for a zero throttle signal would be 1ms / 20ms, which equals 0.05. Therefore, to achieve a zero throttle signal, I multiply the maximum duty cycle by 5 / 1000, or equivalently, divide it by 20.

For the maximum throttle signal, the duty cycle would be 2ms / 20ms, which equals 0.1. Thus, the corresponding maximum throttle signal would be the maximum duty cycle divided by 10.

Additionally, I introduce a slight delay to allow the ESC to register the zero throttle signal. I can confirm this works because I hear specific beeps in a distinct melody.

rprintln!("Zero signal");
ch1.set_duty(ch1.get_max_duty() / 20);


rprintln!("Slight delay");
delay.delay_ms(10000_u32);

let min_limit = max_duty * 5 / 1000;
let max_limit = max_duty / 100;

{% codeblock lang:rust %}
for value in 0..50 {
    //ch1.set_duty(map(value, 0, 100, min_limit, max_limit));
    let throttle = map(value, 0, 100, min_limit, max_limit);
    ch1.set_duty(throttle as u16);
    delay.delay_ms(12_u32);
}
...
fn map(x: u32, in_min: u32, in_max: u32, out_min: u32, out_max: u32) -> u32 {
    (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
}
{% endcodeblock %}

Remote Controller

For my remote controller, I am using the FlySky FS-IA6B. It comes with the transmitter and a receiver. All I had to do is read any bytes from the receiver. FlySky has something called the IBus protocol, which essentially allows it to send the data of all the channels in 32 byte packets through the UART hardware.

Note A channel in this context refers to each inputs on the transmitter, such as the joysticsk and switches.

To my luck, the stm32f4xx-hal repository already contains an example with UART and RTIC. This example uses a UART interrupt to immediately notify the CPU when the UART has new information.

Unfortunately, getting this to work for my usecase proved to be quite difficult. The UART would constantly get new data so it would starve the other tasks. Instead, I switched to polling the UART for new info and implemented a checksum verification to discard erroneous packets.

Note Interrupts allow hardware to notify the CPU of a specific event. Polling is when the CPU checks on the hardware for any new events. This comes at the cost of additional CPU clock cycles.

    #[task(binds = TIM3, priority = 1, local=[rc_timer, prod_flysky, rx])]
    fn process_received_bytes(ctx: process_received_bytes::Context) {

        ctx.local.rc_timer.clear_all_flags();
        ctx.local.rc_timer.start(15.millis()).unwrap();
        
        let rx = ctx.local.rx;

        rprintln!("process received bytes");
            

        let mut bytes = [0u8; 32];
        let mut index = 0;
        let mut checksum: u16 = 0;

        while index != 32 {
            if let Ok(byte) = rx.read() {
                if index == 0 && byte != 32 {
                    // Ignore bytes until the first byte is 32 (space character)
                    continue;
                }
                bytes[index] = byte;
                index += 1;
            }
        }

        //rprintln!("bytes: {:?}", bytes);
        let mut channel_values: [u16; 16] = [0; 16];

        for i in (0..bytes.len()).step_by(2) {
            if i + 1 < bytes.len() {
                // Extract two bytes from the pair
                let byte1 = u16::from(bytes[i]);
                let byte2 = u16::from(bytes[i + 1]);

                if i < 30 {
                    checksum += byte1 + byte2;
                }
                
                // Combine the bytes by multiplying the second number by 256
                //let combined_value = u16::from(byte1) + u16::from(byte2) * 256;
                let combined_value = byte1 | byte2 << 8;

                let channel_index = i / 2;
                //rprintln!("index: {:?}", channel_index);
                // Do something with the combined value (e.g., print or use it)

                channel_values[channel_index] = combined_value;
                //rprintln!("Combined Value: {}", combined_value);
            } else {
                break
            }
        }

        if 65535 - channel_values[15] != u16::from(checksum) {             
            rprintln!("{:?} != {:?}", 65535 - channel_values[15], checksum);                            
        } else {
        
            //rprintln!("Channels: {:?}", channel_values);
            let mut _throttle = channel_values[3] as f32;
            let mut _desired_roll = map(channel_values[1] as f32, 1000., 2000., -15., 15.);
            let mut _desired_pitch = map(channel_values[2] as f32, 1000., 2000., -15., 15.);

            match ctx.local.prod_flysky.enqueue([_throttle, _desired_roll, _desired_pitch]) {
                Ok(()) => {
                    //rprintln!("Data sent");
                }

                Err(_err) => {
                    // Other errors occurred, handle them appropriately
                    // Example: println!("Error occurred while enqueueing data: {:?}", err);
                    rprintln!("Flysky failed to enqueue");
                }
            }
        }
        
    }

PID

A pid is a control loop concept that can use feedback to adjust the inputs to drive the system towards a desired state. I will be simplifing how PIDs work a lot as I probably won’t do a good job at explaining its complexities.

For my drone, I am able to get the attitude of my drone, and I can set the desired attitude with my remote controller. I need some sort of mechanism to force the drone to match the attitude set by the remote controller. A PID controller can do so by constantly adjusting the speed of each motor to match the desired angle. A PID must be configured to work properly by tuning its three parameters; proportional, controller, and integral.

Proportion

The porportional or P-controller takes a fraction of the error. For example, the pitch of my drone may be 15 degrees, but my desired pitch could be 0 degrees. The error would be 15 degrees. The proportion would multiply this error by a factor to adjust the input throttle to each motor.

Integral

The integral or I-controller helps with small errors by taking a fraction of the integral of the error.

Derivative

This controller takes into account on how rapidly the error is changing by taking a fraction of the error’s derivative.

RTIC

RTIC, as mentioned previously, is a real-time framework. In the context of embedded software, a real-time system is defined as one that is deterministic and meets specific timing requirements. This is in contrast to systems like Arduinos, which rely on one large loop. While Arduino-based quadcopters have been developed before, using a real-time framework like RTIC can provide greater reliability in certain scenarios. Determining which approach is better depends on the use case.

RTIC includes a scheduler that determines which task to run by programming the MCU’s hardware. In my project, I have three tasks. The first task polls the UART for new data. The second task employs the Kalman Filter to determine the drone’s attitude from the IMU data. The third task is the flight controller task, where I use a PID controller to generate the appropriate throttle signals for each motor. Each task operates as an interrupt service routine of a timer, allowing them to run approximately every 15ms with high accuracy. RTIC also includes an init task, responsible for configuring all the hardware and setting initial values.

#![no_main]
#![no_std]
#![feature(type_alias_impl_trait)]

use panic_halt as _;
mod kalman;
mod pid;

use rtic::app;

#[app(device = stm32f4xx_hal::pac, dispatchers = [SPI1, SPI2])]
mod app {
    use core::f32::consts::PI;
    use heapless::spsc::{Consumer, Producer, Queue};
    use rtic_monotonics::{create_systick_token, systick::Systick};
    use stm32f4xx_hal::{self as hal};
    

    use hal::{
        i2c::{I2c1, Mode}, 
        pac::{TIM1, TIM2, TIM3, TIM4, TIM5, USART1}, 
        prelude::*, rcc::RccExt, 
        serial::{Config, Rx, Serial}, 
        timer::{self, Channel1, Channel2, Event, PwmChannel}
    };

    use rtt_target::{rprintln, rtt_init_print};

    /** Kalman Filter Libraries **/
    use libm::{atanf, sqrtf};
    use crate::kalman::KalmanFilter;
    use crate::pid::Pid;
    use lsm6dsox_driver::Lsm6dsox;

    const FC_DELTA: u32 = 15;
    const KALMAN_DELTA: u32 = 15;

    #[shared]
    struct Shared {
    }

    #[local]
    struct Local {
        /** FlySky Buffer **/
        rx: Rx<USART1, u8>,
        prod_flysky: Producer<'static, [f32; 3], 10>,
        con_flysky: Consumer<'static, [f32; 3], 10>,
        rc_timer: timer::CounterMs<TIM3>,


        /** Kalman Filter Variables **/
        kalman_timer: timer::CounterMs<TIM4>,
        imu: Lsm6dsox<I2c1>,
        x_kalman: KalmanFilter,
        y_kalman: KalmanFilter,

        prod_kalman: Producer<'static, [f32; 2], 5>,
        con_kalman: Consumer<'static, [f32; 2], 5>,

        /** Flight Controller variables **/
        fc_timer: timer::CounterMs<TIM5>,

        /** ESC **/
        m1: PwmChannel<TIM1, 0>,
        m2: PwmChannel<TIM1, 1>,
        m3: PwmChannel<TIM2, 0>,
        m4: PwmChannel<TIM2, 1>,

        pitch_pid: Pid,
        roll_pid: Pid,
    }


    #[init(local = [
        rx_pool_memory: [u8; 400] = [0; 400],
        queue_kalman: Queue<[f32; 2], 5> = Queue::new(),
        queue_flysky: Queue<[f32; 3], 10> = Queue::new(),
    ])]
    fn init(cx: init::Context) -> (Shared, Local) {

        rtt_init_print!();
        rprintln!("Init");

        let (prod_kalman, con_kalman) = cx.local.queue_kalman.split();
        let (prod_flysky, con_flysky) = cx.local.queue_flysky.split();        

        let token = create_systick_token!();
        Systick::start(cx.core.SYST, 100_000_000, token);

        let dp: hal::pac::Peripherals = cx.device;

        let rcc = dp.RCC.constrain();

        //let clocks = rcc.cfgr.freeze();
        //let clocks = rcc.cfgr.hclk(8.MHz()).freeze();
        let clocks = rcc.cfgr.use_hse(25.MHz()).sysclk(36.MHz()).hclk(25.MHz()).freeze();
        // let clocks = rcc.cfgr
        //     .use_hse(8.MHz())
        //     .sysclk(36.MHz())
        //     .pclk1(36.MHz())
        //     .freeze();

        let gpioa = dp.GPIOA.split();
        let rx_pin = gpioa.pa10; // RX

        // Configure USART1 with a baud rate of 115200
        let rx: Rx<USART1, u8> =
            Serial::rx(dp.USART1, rx_pin, Config::default().baudrate(115200.bps()), &clocks).unwrap();

        let mut rc_timer = dp.TIM3.counter_ms(&clocks);
        rc_timer.start(10000.millis()).unwrap();
        rc_timer.listen(Event::Update);

        let gpiob = dp.GPIOB.split();

        //// LSM6DSOX IMU \\\\

        let scl = gpiob.pb6.into_open_drain_output();
        let sda = gpiob.pb7.into_open_drain_output();

        let i2c = dp.I2C1.i2c(
            (scl, sda),
            Mode::Standard {
                frequency: 104.kHz(),
            },
            &clocks,
        );

        let mut imu = Lsm6dsox::new(i2c).unwrap();
        rprintln!("LSM6DSOX IMU initialized");
        let id = imu.read_id().unwrap();
        rprintln!("id is {:#b}: ", id);

        imu.configure_accel().unwrap();
        imu.configure_gyro().unwrap();

        // Kalman Filter
        let x_kalman = KalmanFilter::new();
        let y_kalman = KalmanFilter::new();

        // Kalman Filter Timer
        let mut kalman_timer = dp.TIM4.counter_ms(&clocks);
        kalman_timer.start(10000.millis()).unwrap();
        kalman_timer.listen(Event::Update);

        // Motor Timer Setup
        let channels = (Channel1::new(gpioa.pa8), Channel2::new(gpioa.pa9));
        let channels_1 = (Channel1::new(gpioa.pa0), Channel2::new(gpioa.pa1));
        
        let pwm = dp.TIM1.pwm_hz(channels, 50.Hz(), &clocks).split();
        let pwm_1 = dp.TIM2.pwm_hz(channels_1, 50.Hz(), &clocks).split();

        let (mut m1, mut m2) = pwm;
        let (mut m3, mut m4) = pwm_1;

        m1.enable();
        m2.enable();
        m3.enable();
        m4.enable();

        rprintln!("Zero signal");
        m1.set_duty(m1.get_max_duty() / 20);
        m2.set_duty(m2.get_max_duty() / 20);
        m3.set_duty(m3.get_max_duty() / 20);
        m4.set_duty(m4.get_max_duty() / 20);

        //rprintln!("m1 Value: {:?}", [m1.get_max_duty() / 20, m1.get_max_duty() / 10]);

        // PID
        let pitch_pid = Pid::new(2.0, 0.1, 0.1);
        let roll_pid = Pid::new(2.0, 0.1, 0.1);

        // Flight Controller Timer
        let mut fc_timer = dp.TIM5.counter_ms(&clocks);
        fc_timer.start(10000.millis()).unwrap();
        fc_timer.listen(Event::Update);

        //process_received_bytes::spawn().ok();

        (
            Shared {  },
            Local {
                rx,
                prod_flysky,
                con_flysky,
                rc_timer,

                imu,
                x_kalman,
                y_kalman,

                prod_kalman,
                con_kalman,
                kalman_timer,

                fc_timer,

                m1, m2, m3, m4,

                pitch_pid,
                roll_pid,
            },
        )
    }

    #[task(binds = TIM3, priority = 1, local=[rc_timer, prod_flysky, rx])]
    fn process_received_bytes(ctx: process_received_bytes::Context) {

        ctx.local.rc_timer.clear_all_flags();
        ctx.local.rc_timer.start(15.millis()).unwrap();
        
        let rx = ctx.local.rx;

        rprintln!("process received bytes");
            

        let mut bytes = [0u8; 32];
        let mut index = 0;
        let mut checksum: u16 = 0;

        while index != 32 {
            if let Ok(byte) = rx.read() {
                if index == 0 && byte != 32 {
                    // Ignore bytes until the first byte is 32 (space character)
                    continue;
                }
                bytes[index] = byte;
                index += 1;
            }
        }

        //rprintln!("bytes: {:?}", bytes);
        let mut channel_values: [u16; 16] = [0; 16];

        for i in (0..bytes.len()).step_by(2) {
            if i + 1 < bytes.len() {
                // Extract two bytes from the pair
                let byte1 = u16::from(bytes[i]);
                let byte2 = u16::from(bytes[i + 1]);

                if i < 30 {
                    checksum += byte1 + byte2;
                }
                
                // Combine the bytes by multiplying the second number by 256
                //let combined_value = u16::from(byte1) + u16::from(byte2) * 256;
                let combined_value = byte1 | byte2 << 8;

                let channel_index = i / 2;
                //rprintln!("index: {:?}", channel_index);
                // Do something with the combined value (e.g., print or use it)

                channel_values[channel_index] = combined_value;
                //rprintln!("Combined Value: {}", combined_value);
            } else {
                break
            }
        }

        if 65535 - channel_values[15] != u16::from(checksum) {             
            rprintln!("{:?} != {:?}", 65535 - channel_values[15], checksum);                            
        } else {
        
            //rprintln!("Channels: {:?}", channel_values);
            let mut _throttle = channel_values[3] as f32;
            let mut _desired_roll = map(channel_values[1] as f32, 1000., 2000., -15., 15.);
            let mut _desired_pitch = map(channel_values[2] as f32, 1000., 2000., -15., 15.);

            match ctx.local.prod_flysky.enqueue([_throttle, _desired_roll, _desired_pitch]) {
                Ok(()) => {
                    //rprintln!("Data sent");
                }

                Err(_err) => {
                    // Other errors occurred, handle them appropriately
                    // Example: println!("Error occurred while enqueueing data: {:?}", err);
                    rprintln!("Flysky failed to enqueue");
                }
            }
        }
        
    }

    /** Mapping Function **/
    
    fn map(x: f32, in_min: f32, in_max: f32, out_min: f32, out_max: f32) -> f32 {
        (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
    }
    


    #[task(binds = TIM4, local=[kalman_timer, imu, x_kalman, y_kalman, prod_kalman], priority = 1)]
    fn sensor_fusion(ctx: sensor_fusion::Context) {
        ctx.local.kalman_timer.clear_all_flags();
        ctx.local.kalman_timer.start(15.millis()).unwrap();

        rprintln!("Sensor fusion");

        let delta_sec = 0.015;

        let accel_data: [f32; 3];
        let gyro_data: [f32; 3];

        let x_accel: f32;
        let y_accel: f32;

        let imu = ctx.local.imu;
        
        accel_data = imu.read_accel().unwrap();
        gyro_data = imu.read_gyro().unwrap();

        y_accel = atanf(accel_data[0] / sqrtf(accel_data[1] * accel_data[1] + accel_data[2] * accel_data[2])) * 180.0 / PI;
        x_accel = atanf(accel_data[1] / sqrtf(accel_data[0] * accel_data[0] + accel_data[2] * accel_data[2])) * 180.0/ PI;

        ctx.local.x_kalman.process_posterior_state(gyro_data[0], x_accel, delta_sec);
        ctx.local.y_kalman.process_posterior_state(gyro_data[1], y_accel, delta_sec);
        
        // [roll, pitch]
        match ctx.local.prod_kalman.enqueue([ctx.local.y_kalman.get_angle(), ctx.local.x_kalman.get_angle()]) {
            Ok(()) => {
                //rprintln!("Data sent");
            }

            Err(_err) => {
                // Other errors occurred, handle them appropriately
                // Example: println!("Error occurred while enqueueing data: {:?}", err);
                rprintln!("IMU Data failed to send");
            }
        }
    }

    #[task(binds = TIM5, 
           local = [fc_timer, con_kalman, con_flysky, 
                    m1, m2, m3, m4,
                    pitch_pid, roll_pid], 
            priority = 1)]
    fn flight_controller(ctx: flight_controller::Context) {
        ctx.local.fc_timer.clear_all_flags();
        ctx.local.fc_timer.start(FC_DELTA.millis()).unwrap();

        let m1 = ctx.local.m1;
        let m2 = ctx.local.m2;
        let m3 = ctx.local.m3;
        let m4 = ctx.local.m4;

        let pitch_pid = ctx.local.pitch_pid;
        let roll_pid = ctx.local.roll_pid;

        let mut kalman_data: [f32; 2] = [0., 0.];
        let mut fly_sky_data: [f32; 3] = [0., 0., 0.];

        rprintln!("FC");

        if let Some(data) = ctx.local.con_kalman.dequeue() {
            kalman_data = data;
            rprintln!("Kalman Data: {:?}", data);
        }
        
        if let Some(data) = ctx.local.con_flysky.dequeue() {
            fly_sky_data = data;
            rprintln!("Flysky Data: {:?}", data);
        } 

        // Error
        let _error_roll =  kalman_data[0];
        let _error_pitch = kalman_data[1];

        // PID
        let _pitch_pid_value = pitch_pid.compute(_error_pitch, FC_DELTA as f32 / 1000.);
        let _roll_pid_value = roll_pid.compute(_error_roll, FC_DELTA as f32 / 1000.);


        let mut throttle: [f32; 4] = [0.; 4];

        throttle[0] = fly_sky_data[0] - _pitch_pid_value - _roll_pid_value; // front left
        throttle[1] = fly_sky_data[0] - _pitch_pid_value + _roll_pid_value; // front right
        //throttle[1] = 1000.;
        
        throttle[2] = fly_sky_data[0] + _pitch_pid_value - _roll_pid_value; // rear left
        throttle[3] = fly_sky_data[0] + _pitch_pid_value + _roll_pid_value; // rear right

        for i in 0..4 {
            if throttle[i] < 1000. {
                throttle[i] = 1000.;
            }
            if throttle[i] > 2000. {
                throttle[i] = 2000.;
            }   
        }
        rprintln!("Pitch PID: {:?}, Roll PID: {:?}", _pitch_pid_value, _roll_pid_value);
        rprintln!("Throttle: {:?}", throttle);


        m1.set_duty(map(throttle[0], 1000., 2000., m1.get_max_duty() as f32 / 20., m1.get_max_duty() as f32 / 10.) as u16); 
        m2.set_duty(map(throttle[1], 1000., 2000., m2.get_max_duty() as f32 / 20., m2.get_max_duty() as f32 / 10.) as u16);
        m3.set_duty(map(throttle[2], 1000., 2000., m3.get_max_duty() as f32 / 20., m3.get_max_duty() as f32 / 10.) as u16);
        m4.set_duty(map(throttle[3], 1000., 2000., m4.get_max_duty() as f32 / 20., m4.get_max_duty() as f32 / 10.) as u16);
    }
}

PCB

I initially used a breadboard for prototyping and testing each component. Afterwards I got each component working together, I designed a PCB to make connections for the ESC, IMU, FlySky Receiver, and the microcontroller on one board.

Current Issues

While the drone is assembled, stable flight has not yet been achieved. I suspect the root cause is the sporadic values from the IMU due to the vibrations from the motor messing with the accelerometer values. As a result of the IMU’s messy values, the PID is unable to get a reliable error to correct on. I’m not yet sure if this is due to my Kalman Filter implementation, if it can yet be improved with some tweaks on the R or Q values, or if I may have to switch to a different Kalman Filter crate such as yakf. This issue can be observed in the video below and the program’s output.

FC
Kalman Data: [-0.90039134, 0.12164825]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -6.5124507, Roll PID: -10.382492
Throttle: [1239.8949, 1219.13, 1226.87, 1206.1051]
Sensor fusion
process received bytes
FC
Kalman Data: [-2.0230548, 0.48110035]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 3.458548, Roll PID: -11.619747
Throttle: [1231.1613, 1207.9218, 1238.0782, 1214.8387]
Sensor fusion
process received bytes
FC
Kalman Data: [1.2617773, 1.6009094]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 10.767213, Roll PID: 24.335114
Sensor fusion
FC
Sensor fusion
FC
FC
Kalman Data: [-2.1917682, 0.08220053]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -1.3731648, Roll PID: -14.535976
Throttle: [1238.9092, 1209.8372, 1236.1628, 1207.0908]
Sensor fusion
process received bytes
FC
Kalman Data: [1.738786, 1.4941175]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 12.501015, Roll PID: 29.591108
Throttle: [1180.908, 1240.0901, 1205.9099, 1265.092]
Sensor fusion
process received bytes
FC
Kalman Data: [-2.5784974, 0.37395483]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -6.6198416, Roll PID: -34.032913
Throttle: [1263.6528, 1195.5869, 1250.4131, 1182.3472]
Sensor fusion
process received bytes
FC
Kalman Data: [0.20860016, 1.2831023]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 8.727188, Roll PID: 18.904137
Throttle: [1195.3687, 1233.177, 1212.823, 1250.6313]
Sensor fusion
process received bytes
FC
Kalman Data: [-0.7358496, 0.118415]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -7.427752, Roll PID: -7.862849
Sensor fusion
process received bytes
FC
Kalman Data: [1.1448559, 1.5381482]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 7.8262134, Roll PID: 19.894714
Throttle: [1195.279, 1235.0686, 1210.9314, 1250.721]
Sensor fusion
process received bytes
FC
Kalman Data: [-2.4780583, 0.006271422]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -10.099969, Roll PID: -29.20888
Throttle: [1262.3088, 1203.8911, 1242.1089, 1183.6912]
Sensor fusion
process received bytes
FC
Kalman Data: [-1.4668975, 0.97744393]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 8.529371, Roll PID: 3.7072783
Throttle: [1210.7633, 1218.1779, 1227.8221, 1235.2367]
Sensor fusion
process received bytes
FC
Kalman Data: [-0.10233903, 0.5720601]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -1.4584388, Roll PID: 8.792378
Throttle: [1215.6661, 1233.2509, 1212.7491, 1230.3339]
Sensor fusion
process received bytes
FC
Kalman Data: [0.4794494, 1.6821591]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 10.864978, Roll PID: 4.738208
Sensor fusion
FC
Sensor fusion
FC
FC
Kalman Data: [-2.4574478, 0.19115067]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -9.944306, Roll PID: -28.328514
Throttle: [1261.2728, 1204.6158, 1241.3842, 1184.7272]
Sensor fusion
process received bytes
FC
Kalman Data: [1.0313929, 1.3718957]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 10.7154255, Roll PID: 25.22327
Throttle: [1187.0613, 1237.5078, 1208.4922, 1258.9387]
Sensor fusion
process received bytes
FC
Kalman Data: [-0.7749517, 0.5282671]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -4.4676566, Roll PID: -13.691817
Throttle: [1241.1594, 1213.7759, 1232.2241, 1204.8406]
Sensor fusion
process received bytes
FC
Kalman Data: [-0.1934669, 0.8849101]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 4.2474403, Roll PID: 3.3897264
Throttle: [1215.3628, 1222.1423, 1223.8577, 1230.6372]
Sensor fusion
process received bytes
FC
Kalman Data: [-1.54417, 0.5238842]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: -1.2590712, Roll PID: -12.1930275
Sensor fusion
FC
Sensor fusion
FC
Kalman Data: [0.26610875, 1.2130305]
Flysky Data: [1223.0, -0.72000027, -5.16]
Pitch PID: 6.4048543, Roll PID: 10.804463
Throttle: [1205.7906, 1227.3995, 1218.6005, 1240.2094]
Sensor fusion
process received bytes