Adapting Causal Encoder–Decoder Models with LoRA

• A new architecture changes where LoRA should go. DeepSeek-V4.1-Flash is a causal encoder–decoder (CED): its 20 encoder blocks write most of a shared KV cache that its 20 decoder blocks read. LoRA trains on it as on earlier models, but an adapter on every block leaves this structure unused: where an adapter goes now decides whether adapters can share a cache.
• The decoder is enough. A rank-16 LoRA on the 20 decoder blocks reached a validation loss close to that of a LoRA on all 40 blocks (0.296 against 0.293 over three seeds) with half the adapter parameters, and beat an encoder-side LoRA of the same size in every seed, by 0.036 to 0.058. Freezing block 20's KV-writing projections as well costs nothing and lets many such adapters share one compressed KV cache at serving time.
• Fewer modules, a frozen ViT, Engram on the CPU. In the full rank-16 all-linear setup, the compressor, the query up-projection, the grouped output projection and the routed experts' down-projection need no LoRA. Freezing the ViT and aligner kept validation loss the same or lower in three of four configurations and cut peak GPU memory by 5.2 GB; keeping the two Engram tables on the CPU saves 22.9 GB per GPU.
• The training support is submitted upstream. Six pull requests to Megatron-LM, Megatron-Bridge, vLLM and veRL make the model trainable and carry LoRA adapters through to rollout.
• In production. Mint Recursive now supports DeepSeek-V4.1-Flash; on a client's proprietary business benchmark, the post-trained model scores 5.96% higher than the official API.

DeepSeek-V4.1-Flash comes with a new model architecture 1. A standard decoder-only transformer is a stack of identical layers, and each layer keeps its own keys and values (KV) for every token it has seen, so a 40-layer model caches 40 sets of them. DeepSeek-V4.1-Flash is still a causal language model that predicts the next token, but it splits its 40 blocks into two halves with different jobs:

• The encoder (blocks 0–19) reads the input and writes most of a compressed global KV cache through Compressed Sparse Attention 2 (CSA2).
• The decoder (blocks 20–39) mostly reads that cache and produces the output.

The cache is shared across blocks. Only four kv_source blocks (2, 8, 14 and 20) write it, and every later block reuses the KV of the nearest source block before it; blocks 0 and 1 use only sliding-window attention. Forty per-layer caches become four shared segments (Figure 1), and each block also keeps a sliding window of its own last 128 tokens. Block 20 is the hinge between the two halves. It is the first decoder block but also the last kv_source block: its compressor turns the encoder's final hidden state into the KV segment that all 20 decoder blocks, itself included, read. What block 20 writes is the decoder's whole view of the encoder, so its KV-writing projections decide whether adapters can share the cache.

Figure 1: From one KV cache per layer to shared KV: a decoder-only transformer (top) and DeepSeek-V4.1-Flash (bottom), where blocks 2, 8, 14 and 20 write four shared segments that later blocks read.

Around this core the model adds an MoE router with an image-specific bias, a ViT with an aligner, manifold-constrained hyper-connections (mHC) 2, two Engram lookup tables of roughly 384M rows each, read at blocks 1 and 14 3, and weights in FP8 and packed MXFP4 (Figure 2).

Figure 2: DeepSeek-V4.1-Flash with the setup this post arrives at: LoRA on decoder blocks 20–39 (0.93B without ViT adapters) with block 20's KV writers frozen. The GB figures are the peak GPU memory saved by freezing the ViT and keeping Engram on the CPU; ÷2 and ÷1 are compression ratios. Inset: block 20, where cmp (the compressor) writes the shared KV; dashed tabs mark adapters the rank-16 all-linear setup can drop, and percentages are shares of that adapter.

LoRA itself carries over. Once the training stack supports DeepSeek-V4.1-Flash, which any fine-tuning needs and which we cover at the end, an adapter on every block trains as it does on earlier models. Applied that way, though, it leaves out what this architecture offers for serving.

On a decoder-only model, an adapter goes on every layer by default, and every adapter needs its own KV cache anyway. Here, placement decides what an adapter costs to serve. An adapter in the encoder changes the shared KV that every later block reads, so each adapter needs a cache of its own. An adapter in the decoder changes only how that KV is read and each block's own 128-token window: with block 20's KV-writing projections frozen, the shared compressed cache stays the base model's, and many adapters can share it. Our Mixture-of-LoRA serving relies on that sharing 4.

That is the core question of this post: where should the adapter go so that it trains well and stays cheap? The decoder is the cheap place, with half the blocks and a shared cache; the question is whether an adapter there learns as well as one on every block. This matters for how we pursue Learning from Real Experience: every specialist we train from real interactions is one more adapter on the same base, so the cheaper an adapter is to train and to serve, the more often a model can absorb new experience. We answer the placement question first, then cover the engineering that makes the model trainable.

Setup

We compared four adapter configurations on Geometry3K 5 (Table 1), each trained with LoRA on the ViT and aligner and again without, three seeds each. The model still sees every image in both cases; only whether the ViT and aligner weights are adapted changes. All runs share the data order, optimizer and parallelism: 4 nodes of 8 H20 GPUs, BF16, context parallelism 2 and expert parallelism 16, uniform full recompute, optimizer CPU offload, LoRA $\alpha = 2r$ for rank $r$ with merge=False, a maximum length of 4,096 tokens and dynamic batching. Training uses 1,472 problems from the Geometry3K train split and validation uses 224 from its validation split, after removing every problem whose diagram also appears in another split. We report validation loss averaged over the evaluations at steps 50, 70 and 100.

All runs use a learning rate of 5e-5 for 100 steps, chosen for its plateau. A rate of 2e-4 descends faster at first (0.42 to 0.54 by step 10, against 1.1 to 1.5 for 5e-5), which is why an 8-step scan from 1e-5 to 2e-4 favored it; over 100 steps, 5e-5 plateaus 0.006 to 0.044 lower in all four configurations.

Table 1: Adapter configurations and their cost over the 100-step runs with ViT adapters. Trainable parameters are the fp32 master bytes divided by four. The rank-8 configuration halves the rank everywhere, the ViT included.

Halving either the depth or the rank halves the fp32 master, but peak GPU memory moves by only 0.6 GB: under optimizer offload, master weights and Adam moments live in host memory, so most of the saving shows up in host memory; the measured median step time falls by 0.78 to 0.79 s for the half-depth adapters and 0.56 s for rank 8.

Adapting the decoder is enough

Figure 3: Training loss (left; thin: every step, thick: 10-step average) and validation loss (right) of the four configurations with ViT adapters, seed 0.

All four runs drop fast and then plateau: validation loss reaches about 0.33 to 0.43 by step 30 (Figure 3, right). All-linear rank 16 descends fastest early on and ends with the lowest training loss (Figure 3, left), as its extra capacity should allow, but on the validation plateau the decoder-half adapter catches up with it at half the trainable parameters (Table 2, Figure 4). So does all-linear rank 8, the other way to halve the budget. At the same cost, the decoder half beats the encoder half in every seed (Figure 5): by 0.036 to 0.048 with ViT adapters and 0.052 to 0.058 without. Where the half goes matters more than how large the adapter is.

Table 2: Validation loss averaged over the step-50, 70 and 100 evaluations: mean over three seeds, with the range across seeds in parentheses.

Figure 4: Where the adapter goes decides whether the shared KV changes; LoRA on blocks 20–39 with block 20's KV writers frozen keeps one base cache for N adapters. Right: trainable parameters, validation loss (Table 2) and held-out test loss, with ViT adapters.

Three further checks look beyond this 100-step validation plateau:

• Held-out test. On the 601 test problems, on which nothing was tuned (one retrained seed per configuration), the decoder half reaches a lower test loss than all-linear rank 16: 0.190 against 0.201 without ViT adapters, and 0.193 against 0.210 with them. Rank 8 is at 0.183 and 0.191, and the encoder half is at 0.312 and 0.287.
• Longer training. Run to 300 steps without ViT adapters (one seed), every configuration reached its best point between steps 90 and 180; afterwards rank 8 and the decoder half overfit less than the full adapter. At their best points rank 8 (0.246), all-linear rank 16 (0.249) and the decoder half (0.253) are within 0.007, with the encoder half at 0.330. That run's own 100-step plateau, 0.02 to 0.05 above those minima, gives the same ranking.
• A second task. On A2UI, a text-only task from our Macaron agent in which replies combine natural language with JSON that renders interface components, at about twenty times the tokens per step, plateau validation loss was 0.189 for all-linear rank 16, 0.193 for rank 8 and 0.192 for the decoder half (one seed, no ViT adapters).

Figure 5: Validation loss (mean of steps 50, 70 and 100) with (filled) and without (hollow) ViT adapters: mean over three seeds (dot) and range (line).

Why the decoder

Two properties fit what we measured. First, the encoder of a causal encoder–decoder holds the memory: three of the four blocks that write compressed KV sit in it, and the fourth, block 20, builds the decoder's KV from the encoder's output. An encoder adapter perturbs that shared memory for the whole model, while a decoder adapter mostly changes how it is read. Second, pruning studies find the deeper layers of pretrained language models more redundant than the shallow ones 6, which leaves them room to take on a new task while the shallow layers keep what they store. Decoder blocks also sit closest to the loss, behind the shortest Jacobian chain.

The ranking is robust to the encoder's learning rate: the encoder half receives more gradient energy (the sum of squared gradient norms), and halving its rate or applying per-block decay leaves the ranking unchanged.

Sharing one cache across decoder-half adapters also needs block 20's K/V and compressor projections frozen, and freezing them cost nothing (plateau loss moved by −0.001, −0.008 and −0.002 across three seeds).

The ViT can stay frozen

Dropping the ViT and aligner adapters lowered validation loss for both all-linear adapters, left the decoder half unchanged and raised it only for the encoder half (Table 2). Training also got cheaper: steps ran 3 to 9% faster and peak GPU memory fell by 5.2 GB in every configuration, from 87.2–87.8 GB to 82.0–82.6 GB. The saving comes from the backward pass: with nothing trainable in the ViT and aligner, it stops at the language model, so the ViT's activations no longer have to be kept, and its backward and recompute work drops out of each step (Figure 6). The 9M ViT and aligner adapter parameters themselves are negligible.

Figure 6: With ViT adapters, the backward pass runs through the aligner and the ViT and needs their activations; with them frozen, it stops at the language model. Activation tiles are schematic.

Once the ViT is frozen as well, the decoder half needs less work than Table 1 suggests: nothing before block 20 trains, so the backward pass can stop there. With or without recompute, the encoder's activations can be dropped, and the backward pass runs through only the 20 decoder blocks, halving its compute. Megatron-LM #8664 keeps this true across pipeline stages, where a later stage's input always requires a gradient.

Freezing the ViT also makes runs more repeatable: plateau loss varied across seeds by 0.003 to 0.012 without ViT adapters and by 0.015 to 0.052 with them. A tenth of the learning rate for the ViT adapters closes most of the gap: on all-linear rank 16, step-100 validation loss averaged 0.273 with a 0.022 range, against 0.289 and 0.054 at the shared rate and 0.274 and 0.014 with the ViT frozen (Figure 7). That fits a learning rate too large for them, as the next section explains; freezing reaches the same loss at lower cost.

With the ViT frozen, the model still reads the image. After 100 steps, giving each validation problem another problem's diagram raised validation loss by 0.66 for all-linear rank 16 with and without ViT adapters alike, and by 0.51 to 0.63 for the other configurations, while the base model's loss barely moved (−0.06; one seed each). The language-side adapters learn to read the frozen image features.

Which modules need LoRA

Figure 7: Step-100 validation loss of module-level variants: mean over three seeds (dot), each seed (circle) and range (line); dashed lines mark each group's adapter on all modules.

We trained adapters on several subsets of the linear modules, all without ViT adapters; the text quotes plateau averages, while Figure 7 shows step-100 values. The indexer needs no adapter: top-k selection passes no gradient to it, so an adapter there stays at zero. The first subset leaves out three attention-side modules: the query up-projection (wq_b), the grouped output projection (wo_a) and the compressor. The query and output projections are each factorized in two steps (wq_a then wq_b, and the grouped wo_a then wo_b), so an adapter on one step of each still reaches the whole projection, and the compressor exists only in the four kv_source blocks. wo_a and the compressor are also the two projections that call their GEMMs directly and needed new adapter code in vLLM. This trimmed-attention set matched all-linear, at rank 16 or at rank 32 with $\alpha = 32$: plateau loss 0.277 and 0.279 against 0.285. The three modules hold 1.6% of the parameters, and the result shows that the compressor needs no adapter. On the decoder half the trimmed set did slightly better (0.290 against 0.296); at the smallest budget, all-linear rank 8, the full set stays ahead (0.272 against 0.281).

The routed experts hold 96% of the adapter, and their two projections split sharply. Dropping the down-projection (33% of the parameters, 58% of the gradient energy over an 8-step run) removed a third of the adapter (1.77B to 1.20B parameters, counted in the saved adapter files; Table 1 counts the fp32 master) at no cost (0.280 against 0.285); the saving is in optimizer state, while GPU memory and step time stay about the same. On the decoder half, dropping it on top of the trimmed-attention set gave 0.302, level with all modules (0.296) within the spread across seeds, though a little above the trimmed set alone (0.290). The gate/up projection (63% of the parameters, 5% of the gradient energy) is the one to keep: with it, step-100 loss is 0.274 with a 0.014 spread across seeds, against 0.293 and 0.067 without. The attention and shared-expert adapters, about 3% of the parameters, carry much of what the adapter learns: on the same blocks, an adapter on all modules beats routed-expert-only adapters by 0.05 in plateau loss (gate/up and down), 0.07 (gate/up only) and 0.18 (down only).

Figure 8: Left: each module group's share of the all-linear rank-16 adapter's parameters and gradient energy. Right: change in validation loss when that group's LoRA is zeroed.

Zeroing one module group at a time shows directly which modules a trained adapter relies on (Figure 8). We zeroed each group's LoRA in a separately trained all-linear rank-16 adapter with ViT adapters (100 steps, seed 0, validation loss 0.265) and re-ran validation: the attention output projections, with 0.9% of the parameters, mattered most (+0.018), and the routed experts' down-projection, with 58% of the gradient energy, left the loss where it was (−0.001). Gradient share and importance are separate quantities here, because Adam normalizes each parameter's update. Each group alone moved the loss by at most 0.018, while zeroing all of them returns the base model's 6.14: what the adapter learns is spread across modules, and the rest can make up for a module left untrained, as the trimmed-attention set shows for the grouped output projection.

Divided by a module's share of the parameters, though, gradient energy measures something that survives Adam's normalization:

where $G_c/G$ and $n_c/n$ are module $c$'s shares of the gradient energy and of the parameters, so $\rho_c$ is the mean squared gradient per parameter in module $c$, relative to the whole adapter. If a module's output is $\kappa$ times more sensitive to its weights than average, its gradient is $\kappa$ times larger and its curvature $\kappa^2$ times larger, so the step that would minimize the loss shrinks to $1/\kappa$ of the average, while Adam's step stays near the learning rate. A shared learning rate is then about $\kappa \approx \sqrt{\rho_c}$ times too large for that module, the same reasoning that leads LoRA+ to give LoRA's two matrices different learning rates 7 and muP to scale learning rates per layer 8. The ViT stands out at $\rho_c \approx 63$, an effective step about 8 times the average, and is where a tenfold smaller learning rate behaved as predicted. Language-model modules sit at about 7 or below, except the compressor's KV projection at 17, which needed no adapter anyway. The ratio works as a flag for modules that need a smaller learning rate.

For the language model, 5e-5 already sits near the top of the usable range on this task, so one shared rate serves every language-model module: it beats LoRA+ with the learning rate of LoRA's up-projection $B$ at 4 or 16 times that of its down-projection $A$ (0.285 against 0.301 and 0.344), and matches LoRA+ with $A$'s rate lowered to 1.25e-5.

Most of this work applies to any fine-tuning of DeepSeek-V4.1-Flash, LoRA or not. The model breaks assumptions that a training stack makes for every run: that recompute is layer-local, that padding is harmless, that every parameter belongs on the GPU and that the released checkpoint loads as is. The LoRA-specific part is making the inference engine apply adapters wherever the trainer puts them. The changes are tracked in six upstream pull requests (Figure 9).

Figure 9: Where training DeepSeek-V4.1-Flash with LoRA needed changes. Numbered badges mark the six upstream PRs in the order listed at the end of this post; the vLLM rollout (dashed) is used for RL only.

CSA2 carries state across layers, so recompute has to carry it too

A CSA2 block reads, and in a source block writes, a compressed global KV that later blocks depend on:

Here $h_l$ is the hidden state entering block $l$, $s_l$ the CSA2 state it reads (shared KV, index and candidate blocks), $T_l$ the block's update of that state and $F_l$ the rest of the block. Layer-local activation checkpointing assumes that re-running layer $l$ needs only its input $h_l$. Under CSA2 it also needs $s_l$, so a checkpoint that drops the state recomputes the wrong function. Megatron-LM #8018 adds a stateful CSA2 wrapper to Megatron's hybrid stack that carries the shared KV, the index, the candidate blocks and the mHC state through full recompute.

The same PR makes CSA2 work on packed (THD) sequences and under context parallelism. Compression groups, validity masks and rotary positions come from the logical sequence layout, so padding stays out of compression and sparse selection. Across context-parallel ranks, each rank exchanges a halo at its partition boundary and compresses its own span, and every global KV block has one owner (Figure 10).

Figure 10: CSA2 state has to travel with the hidden state (Megatron-LM #8018): (a) through recompute, (b) in packed sequences, where compression groups follow each sequence, and (c) under context parallelism, where rank 1 receives a halo of boundary rows. Sizes are schematic.

Modality routing has to drop padding and keep shapes fixed

V4.1-Flash routes text and image tokens with one router but different expert biases:

We freeze both biases during fine-tuning, keeping the bias terms identical in rollout and training. Padded rows should contribute neither routing probability nor load-balancing statistics, yet the dispatcher expects every row to be routed. Megatron-LM #7763 keeps dispatcher rows for padded tokens but zeroes their routing probabilities and leaves them out of the load-balancing counters, so tensor shapes stay fixed.

Engram stays on the CPU

The two Engram tables, read at blocks 1 and 14, each hold roughly 384M rows of 256 dimensions. They are frozen during LoRA training and read only through sparse lookups, so we keep them in CPU pinned memory and gather the rows each micro-batch needs. Compared with keeping them on the GPU, this saves 22.9 GB of peak memory per GPU (106.8 to 83.9 GB, measured on a half-depth adapter in the short learning-rate scan) at the same step time. The work is in keeping them there: generic training code, such as DDP parameter sync, treats every parameter as GPU-resident. Megatron-LM #7762 adds the pinned-CPU row lookup and has DDP parameter sync skip these CPU-resident parameters, while the lookup's routing metadata stays on the GPU.

The released checkpoint has to round-trip

Megatron-Bridge #6199 maps the released Hugging Face checkpoint onto the native Megatron-Core model, including the hyper-connections, the ViT, the aligner and the Engram tables. It decodes the block-32 FP8 weights and packed MXFP4 experts on import, restores both layouts on export, and streams Engram rows in bounded chunks in both directions: from SafeTensors straight to their owning ranks on import, and out of those ranks to vLLM on export, without assembling a full table. It also lets LoRA target the grouped output projection, which the model keeps as a bare parameter, so its adapter trains and exports like any other.

Rollout has to see the adapter the trainer updates

In RL the trainer pushes base weights and adapters to vLLM separately (merge=False), so vLLM must apply the adapter everywhere the trainer put one. This takes three additions: the multimodal wrapper needs LoRA registration, and the grouped attention output projection and the compressor, which call their GEMMs directly, need adapter paths. vLLM #57245 adds LoRA to the grouped output projection, for BF16 and FP8 base weights, and to the compressor; it also registers the ViT and its connector (the aligner) with adapter state scoped to each image. veRL #7483 then makes the unmerged sync itself correct: it probes how the target model's loader names wrapped base weights, and it unwraps the quantization wrapper that LoRA installs over the MXFP4 MoE method so the post-load repack still runs.

Router replay has to skip image rows

In RL, vLLM generates the rollout with its own router and Megatron recomputes it with another. Rollout Routing Replay (R3) 9 replays vLLM's expert choices in the Megatron forward pass; we described its general fixes in an earlier post 10. V4.1-Flash adds a multimodal twist. vLLM records routes on the compact sequence, where an image is a single placeholder, while Megatron runs on the image-expanded sequence, and the router adds an image-specific bias. A route recorded at the placeholder is therefore invalid for the expanded image tokens, so we replay text rows and route image rows natively (Figure 11; veRL #7483).

Figure 11: Router replay with images: text rows replay their recorded top-6 experts, while image tokens, which vLLM records as a single placeholder, are routed natively with the image bias.

The experiments answer the placement question: adapting the decoder is enough. A rank-16 LoRA on decoder blocks 20–39 reached a validation loss of 0.296, within 0.003 of the 0.293 reached on all 40 blocks, with half the adapter parameters, and beat an encoder-side LoRA of the same size in every seed, by 0.036 to 0.058; freezing block 20's KV-writing projections on top cost nothing. The result held on held-out test problems, over 300 training steps and on a second, text-only task.

Done this way, adapters become cheap enough to use at scale. In practice:

• Adapt the decoder only. Freeze block 20's KV-writing projections so that all adapters share one compressed KV cache. All-linear rank 8 is the other half-budget option that held up.
• Freeze the ViT and aligner. Leaving out their adapters keeps validation loss the same or lower in three of four configurations, makes steps 3 to 9% faster and saves 5.2 GB of peak GPU memory. The model still reads the image. If you do train them, give them a tenth of the learning rate.
• Trim by ablation. In the full rank-16 all-linear setup, the compressor, query up-projection, grouped output projection and the routed experts' down-projection, a third of the parameters, can all go. Keep the routed experts' gate/up projection and the small attention and shared-expert adapters.
• Budget host memory. With optimizer offload, adapter size shows up in host memory and optimizer-step time; peak GPU memory barely moves. Keep the Engram tables on the CPU as well.

Together, these make adapters cheap enough to serve Experiential Intelligence: new experience can arrive as many small decoder-side adapters over one shared base and cache, each far cheaper than a retraining.

Mint Recursive, Mind Lab's serverless and automated model training platform 11, now supports DeepSeek-V4.1-Flash. We use it to post-train DeepSeek-V4.1-Flash for financial clients. On a client's proprietary business benchmark, the post-trained model scores 5.96% higher than the official DeepSeek-V4.1-Flash API.

• [Megatron-LM] Extend DeepSeek V4.1 CSA2 with THD, context parallel, and full recompute #8018
• [Megatron-LM] Fix modality MoE padding routing and selective recompute #7763
• [Megatron-LM] Keep Engram CPU lookup parameters out of generic device synchronization #7762
• [Megatron-Bridge] Add DeepSeek-V4.1 Flash support #6199
• [vLLM] Add DeepSeek V4.1 text and vision adapter support #57245
• [veRL] Support LoRA for DeepSeek V4, DeepSeek V4.1 and GLM-5.3 Flash #7483
• [Megatron-LM] Agree on frozen-prefix gradient boundaries across pipeline stages #8664

[1] DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression (DeepSeek-AI et al, 2026)

[2] mHC: Manifold-Constrained Hyper-Connections (Xie et al, 2025)

[3] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (Cheng et al, 2026)

[4] Macaron-V1-Preview: 749B MoL Agent Model post-trained from GLM5.1 (Mind Lab et al, 2026)

[5] Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning (Lu et al, 2021)

[6] The Unreasonable Ineffectiveness of the Deeper Layers (Gromov et al, 2024)

[7] LoRA+: Efficient Low Rank Adaptation of Large Models (Hayou et al, 2024)

[8] Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (Yang et al, 2022)

[9] Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers (Ma et al, 2025)

[10] Router Replay R3: Why It Failed and How We Fixed It (Mind Lab et al, 2026)

[11] Introducing Macaron-V1.1 and Mint Recursive (Mind Lab et al, 2026)

Mind Lab

Core Contributors

Steven Chiang, Nolan Ho, Atlas Zeng, Nemo Lv, Bosong Yang, Andrew Chen, Pony Ma

Team

Amy Chen, Jingwei Cao, Song Cao, Andrew Chen, Kaijie Chen, Cleon Cheng, Steven Chiang, Hera Feng, Jun Gao, Haonan He, Nolan Ho, Ori Hong, Xinyu Hu, Bruce Jin, Fancy Kong, Andrew Lei, Junjie Li, Lucian Li, Miles Li, Smith Li, Theo Li, Wish Liao, Kieran Liu, Lawrence Liu, Ryan Liu, Yuxin Lu, Nemo Lv, Pony Ma, Minyu, Peixuan, Taotao Qian, Luozheng Qin, Aiden Tang, Nolan Weng, Wynn, Jinxiang Xie, Bosong Yang, Rio Yang, Regis Ye, Young Ye, Albert Yin, Josh Ying, Atlas Zeng, Martin Zeng, Anya Zhang, Harry Zhang, Jason Zhang, Dewu Zheng, Adrian Zhou, Yuhua Zhou, Juno Zhu, Yuhan Zhu and Mindverse Team

Names are listed alphabetically within team.