[Draft] How Much Precision Do We Need? Quantifying HDC-on-CiM Noise Tolerance and Energy Savings¶
Abstract¶
Compute-in-Memory (CiM) and Hyperdimensional Computing (HDC) offer complementary paths toward energy-efficient machine learning. CiM minimizes data movement by performing analog vector–matrix multiplications in situ, while HDC provides algorithmic robustness under low precision and noise. This paper quantifies how far CiM precision can be reduced for HDC classifiers before accuracy breaks down, and what energy savings this enables. We build a compact HDC-on-CiM simulator that injects additive and multiplicative analog noise and emulates analog-to-digital converter (ADC) quantization using a train-learned global scale (fit once at 8 bits, then evaluate at 3–8 bits) and per-dimension quantization, averaging over multiple random seeds and random projections to obtain smooth, comparable accuracy–noise–energy curves. On the 10-class MNIST benchmark, HDC accuracy remains remarkably stable across both noise and precision. Under additive Gaussian noise up to \(\sigma = 0.20\), accuracies for 3–8 ADC bits stay within approximately \(0.3\) percentage points of the 8-bit baseline, while multiplicative noise of similar magnitude has an even smaller effect. In an ADC-dominated energy regime, moving from 8 to 4–5 bits reduces modeled per-inference energy by roughly \(65–70\%\) while preserving MNIST accuracy within this narrow band. These results identify practical precision–energy operating points for CiM–HDC co-design and illustrate how algorithmic robustness can be directly traded for circuit-level energy savings.
Additional Key Words and Phrases¶
Compute-in-Memory (CiM), Analog In-Memory Computing (AIMC), Hyperdimensional Computing (HDC), Low-Precision Inference, ADC Quantization, Quantization Noise, Noise Tolerance, Precision Scaling, Energy–Accuracy Trade-off, MNIST.
Introduction¶
AI’s computational appetite is rising quickly, and energy is the constraint that increasingly matters. Much of the cost comes from moving data rather than arithmetic itself: the von Neumann bottleneck makes shuttling activations and weights between memory and compute the dominant energy budget in many workloads.
Compute-in-Memory (CiM) architectures attack this bottleneck by performing analog multiply–accumulate (MAC) operations inside memory arrays, reporting large, sometimes order-of-magnitude, energy gains compared to conventional digital accelerators in prototype systems123. However, real devices bring non-idealities such as thermal noise, IR-drop, and device variation, and the energy and latency overheads of converters (ADC/DAC) can erode headline gains—particularly as precision increases4.
Hyperdimensional Computing (HDC)—also called Vector Symbolic Architectures (VSA)—offers a complementary algorithmic lever. It encodes information in high-dimensional vectors and relies on simple arithmetic operations such as binding, bundling, and permutation. These distributed codes are intrinsically robust to quantization and stochastic perturbations, and they map naturally to vector–matrix products that CiM hardware computes efficiently56.
Prior work has advanced both fronts: programmable CiM fabrics and converter-aware architectures on the hardware side1234, and demonstrations that HDC maintains accuracy with low precision and noise on the algorithmic side567. What remains under-specified is the quantitative map from CiM-style precision and noise to end-to-end accuracy and energy for HDC workloads, particularly on standard benchmarks such as MNIST. Existing studies often rely on single seeds or uncalibrated quantization models, leaving a gap in reproducible, smooth accuracy–noise–precision curves that can guide hardware–algorithm co-design.
This paper makes three contributions. First, we develop a compact HDC-on-CiM simulator with calibrated quantization and parametric energy modeling. Second, we systematically evaluate accuracy–noise–precision trade-offs on MNIST using multi-seed, multi-projection averaging to obtain smooth, reproducible curves. Third, we derive a quantified mapping from ADC bit-depth to energy savings that identifies practical precision operating points for CiM–HDC co-design.
Problem Statement¶
The goal of this project is to quantify how CiM-style analog noise and ADC bit-depth affect the accuracy of an HDC classifier and the corresponding energy per inference, in order to identify practical precision–energy operating points. In other words, we aim to establish how much analog noise and quantization an HDC classifier can tolerate when executed on CiM hardware, and what energy savings these tolerances enable, particularly on standard benchmarks such as MNIST.
This question matters because converter energy often dominates system power at higher precisions. Being able to operate reliably at lower bit-depths directly translates to reduced energy consumption and potentially higher throughput. Determining the quantitative tolerance of HDC models to analog noise and quantization thus enables concrete design rules—such as the minimum ADC precision that preserves baseline accuracy—and informs compiler and runtime policies for adaptive precision scheduling or shared converter allocation.
Answering this question provides three immediate benefits. First, it yields empirical precision guidelines for deploying HDC on CiM hardware, based on smooth and reproducible accuracy–noise–precision curves obtained through multi-seed and multi-projection evaluation. Second, it supports dynamic control of energy–accuracy trade-offs by adjusting ADC precision in response to workload requirements. Finally, it establishes a reproducible baseline for extending the model to more detailed physical phenomena such as IR-drop correlations, temporal drift, and device variability.
Technical Approach¶
To explore the quantitative relationship between analog noise, ADC precision, and classification accuracy in HDC systems, we developed a compact and transparent HDC-on-CiM simulator. The simulator models three key aspects of a CiM system: (1) the encoding and inference process of an HDC classifier, (2) the effect of analog non-idealities such as additive and multiplicative noise and limited converter precision, and (3) the corresponding energy cost per inference. Each component is intentionally simplified to isolate the most salient interactions between algorithmic robustness and hardware constraints, providing a reproducible baseline for future analog-aware co-design studies.
HDC Encoding and Inference Model¶
At a high level, an HDC classifier represents each data point as a high-dimensional hypervector that distributes information uniformly across its dimensions. This distributed representation makes HDC particularly resilient to individual bit flips or analog variations. In this project, each input feature vector \(x \in \mathbb{R}^{F}\) is projected into a \(D\)-dimensional bipolar hypervector through a random projection matrix \(P \in \{\pm1\}^{D \times F}\) scaled by \(1/\sqrt{F}\):
During training, hypervectors corresponding to samples of the same class are summed and re-binarized to form a class prototype or class hypervector:
At inference time, an unseen input is projected and binarized using the same matrix \(P\), and its similarity to each class prototype is computed using the dot product, which is equivalent to a cosine similarity in the bipolar space. The predicted label corresponds to the class with maximum similarity:
This model abstracts away low-level circuit details while preserving the algorithmic operations that would map naturally to analog in-memory computing: vector–matrix multiplications for encoding, and dot products for similarity search. Both are dominated by simple multiply–accumulate (MAC) operations, which CiM architectures can perform directly within memory arrays.
Modeling Noise and Quantization¶
To emulate CiM non-idealities, the simulator injects controlled levels of analog noise at the MAC outputs. Two types of Gaussian noise are modeled:
Additive noise serves as a first-order, i.i.d. approximation of effects such as IR-drop or thermal variation, while multiplicative noise captures proportional gain variation. These noise models reflect realistic imperfections in analog memory arrays and align well with HDC’s use of distributed codes: small perturbations rarely flip enough dimensions to change the decision, preserving class separability.
Quantization is modeled as uniform \(b\)-bit rounding that mimics ADC at different resolutions. Instead of using an arbitrary scale, we estimate a realistic quantization step size by learning it once from training data at 8 bits and then coarsening it for lower precisions. The simulator first computes the projection outputs \(Y_{\mathrm{tr}} = P X_{\mathrm{tr}}\) and estimates their spread. In the simplest case, we use a single global standard deviation \(\sigma(Y_{\mathrm{tr}})\); alternatively, we can compute a separate standard deviation for each dimension to enable per-dimension quantization. In either case, we define the 8-bit quantization range as
and the corresponding 8-bit step as
For a target bit-depth \(b \in \{3,4,5,6,8\}\), the step size is coarsened as
and quantization is applied before the binarization step:
When per-dimension quantization is enabled, \(\sigma\), \(\text{range}_8\), and \(\Delta_8\) become vectors over dimensions rather than scalars, but the above relationships still hold elementwise. Crucially, the quantization scale is learned once at 8 bits during training and reused across all noise levels and bit-depths. This formulation preserves the statistical structure of analog outputs, explicitly links the bit-depth \(b\) to a measurable quantization step, and avoids retraining at each precision.
Bit-Depth and Hardware Interpretation¶
The parameter \(b\) represents the effective ADC resolution used to digitize analog MAC outputs in a CiM array. Current CiM prototypes typically operate at 6–8 bits to balance accuracy and converter power, while aggressive designs explore 3–5 bits to further reduce energy. In our simulator, we sweep \(b \in \{3,4,5,6,8\}\) to cover both conventional and aggressively quantized regimes.
Lowering \(b\) decreases precision but exponentially reduces converter energy, which we model as scaling roughly with the number of ADC levels: \(E_{\text{ADC}}(b) \propto 2^b\). By combining this scaling with the HDC classification pipeline, the simulator explicitly models the trade-off between accuracy degradation and estimated energy savings as a function of ADC bit-depth.
Energy Model¶
To connect algorithmic precision to hardware cost, we estimate per-sample inference energy as
where \(N_{\text{MAC}} = D(F + C)\) covers both the encoding and similarity computations, and \(N_{\text{ADC}}\) represents the number of analog-to-digital conversions per inference. In our implementation, we conservatively assume one ADC conversion per projected dimension, so \(N_{\text{ADC}} = D\).
The energy values are parameterized by two knobs: the per-MAC energy \(E_{\text{MAC}}\) and the per-conversion energy of an 8-bit ADC, \(E_{\text{ADC}}(8)\). By default, we use \(E_{\text{MAC}} = 0.5\) pJ and \(E_{\text{ADC}}(8) = 10\) pJ, consistent with recent device-level measurements38. The ADC energy at bit-depth \(b\) is then modeled as
reflecting the exponential dependence on resolution. This default regime yields a balanced mix where both MAC and ADC costs contribute to the total energy.
To probe more extreme hardware scenarios, the simulator also supports alternative regimes in which either the memory array or the converters dominate the energy budget. In particular, for the ADC-dominated regime used in our MNIST experiments, we set \(E_{\text{MAC}} = 0.15\) pJ and \(E_{\text{ADC}}(8) = 60\) pJ, pushing the system into a converter-heavy operating point. This configuration emphasizes the impact of reducing ADC bit-depth on total energy, and it is the one used to produce the steep energy savings (\(65–70\%\)) reported in the evaluation.
Implementation and Evolution¶
The simulator is implemented in Python with NumPy for reproducibility and accessibility across disciplines. This choice reflects a deliberate design philosophy: to make the code usable both by hardware engineers interested in analog modeling and by computer scientists working on algorithmic robustness. Over the course of the project, the implementation evolved through a series of versions:
- v1.1: Implemented baseline HDC encoding and a simple per-sample quantization model without train-learned scaling.
- v1.2: Introduced train-learned global scaling to capture realistic ADC behavior, learning an 8-bit range from training projections and reusing it across bit-depths.
- v1.3: Added a parametric energy model linking ADC bit-depth to estimated per-inference energy, enabling energy-–accuracy trade-off analysis.
- v1.4: Integrated unified sweeps across noise and precision levels, multiple datasets (synthetic and
digits), and reproducible plotting for systematic evaluation. - v1.5: Extended the simulator to the 10-class MNIST benchmark via Principal Component Analysis (PCA)-based dimensionality reduction, and added optional per-dimension quantization to better match dimension-wise statistics of high-dimensional projections.
- v1.6: Introduced multi-seed and multi-projection averaging for each \((\sigma, b)\) point, as well as configurable energy regimes (default, ADC-dominated, MAC-dominated). This version adds command-line controls for noise type, dataset, projection dimensionality, and output ranges, and it is the basis for the smooth, reproducible accuracy–noise–energy curves reported in this paper.
Beyond these software refinements, the simulator is designed for extensibility. Future iterations could incorporate measured device statistics (e.g., drift or IR-drop correlations) as noise parameters, emulate software-level calibration strategies to compensate for process variation, or even model associative memory queries directly within analog arrays—eliminating the need for digitization altogether. Together, these extensions would deepen the co-design bridge between hardware and algorithm, aligning analog device behavior with energy-aware computation.
Evaluation¶
Experimental Setup¶
We evaluate the robustness–efficiency trade-offs of our HDC-on-CiM simulator on the 10-class MNIST benchmark, reduced to \(F=128\) features via PCA. Unless otherwise noted, all experiments use projection dimensionality \(D=1024\) and \(C=10\) classes. The model is trained once at 8-bit precision to establish the quantization scale (with per-dimension statistics enabled), then evaluated at bit-depths \(b \in \{3,4,5,6,8\}\) and noise levels \(\sigma \in [0,0.20]\) sampled in 17 steps.
For each \((\sigma,b)\) configuration, accuracy is averaged across three independent random projections and ten independent noise realizations. Both additive and multiplicative Gaussian noise are tested, and results are reported under two energy models: a default balanced regime and an ADC-dominated regime. The primary metrics are classification accuracy and modeled energy per inference.
Additive Noise and Quantization Effects¶
Under additive noise, MNIST accuracy is remarkably insensitive to both noise amplitude and bit-depth. At \(\sigma=0\) and 8 bits, the mean accuracy is approximately \(0.7667\). Reducing precision to 3 bits changes accuracy by only about \(0.003\) in absolute, and all intermediate bit-depths (4–6 bits) lie essentially on top of the 8-bit baseline. At \(\sigma=0.20\), accuracies across 3–8 bits remain clustered between roughly \(0.762\) and \(0.765\), again within a narrow band.
Figure 1 shows accuracy versus additive noise for \(b=4\). Accuracy declines gently from approximately \(0.766\) at \(\sigma=0\) to about \(0.764\) at \(\sigma=0.20\), with error bars reflecting the variability across projections and noise draws. This confirms that the HDC classifier maintains stable performance even under relatively large additive perturbations in the analog domain.

Figure 1. Accuracy versus additive noise amplitude on MNIST at b = 4. Accuracy decreases slowly with σ, remaining within a narrow band around the clean baseline up to σ = 0.20.
Energy–Accuracy Trade-off (Default Regime)¶
We first combine the accuracy results with the default energy model (\(E_{\mathrm{MAC}}=0.5\) pJ, \(E_{\mathrm{ADC}}(8)=10\) pJ). In this balanced regime, ADC energy grows with bit-depth while MAC energy is fixed.
Figure 2 shows the resulting energy–accuracy curve at \(\sigma=0.10\). Moving from 3 to 8 bits increases energy from approximately \(70\,\text{k}\)pJ to about \(81\,\text{k}\)pJ, while accuracy remains effectively constant (all points lie within approximately \(0.003\) of each other). In particular, reducing precision from 8 to 4 bits lowers modeled energy by about \(12\%\) with negligible accuracy change. This validates the qualitative intuition that HDC’s robustness can be exploited to shave converter precision, even in relatively conservative energy settings.

Figure 2. Energy–accuracy trade-off on MNIST under additive noise (σ = 0.10) in the default energy regime. Energy increases with bit-depth, but accuracy is nearly flat across 3–8 bits.
ADC-Dominated Regime¶
To highlight scenarios where converter power dominates, we repeat the additive-noise sweep with an ADC-dominated configuration, setting \(E_{\mathrm{MAC}}=0.15\) pJ and \(E_{\mathrm{ADC}}(8)=60\) pJ. This exaggerates the cost of additional bits while leaving the HDC algorithm unchanged.
In this regime, lowering precision from 8 to 4–5 bits produces dramatic energy savings: energy drops by roughly \(65\)–\(70\%\) while accuracy stays within approximately \(0.002\) of the 8-bit baseline. The corresponding energy–accuracy curve (Figure 3) therefore shows a broad Pareto-optimal region at low bit-depths, with almost no penalty in classification performance.

Figure 3. Energy–accuracy trade-off on MNIST under additive noise (σ = 0.10) in the ADC-dominated regime (EMAC = 0.15 pJ, EADC(8) = 60 pJ). Reducing bit-depth from 8 to 4–5 bits yields about 65–70% lower energy with essentially unchanged accuracy.
Multiplicative Noise¶
Multiplicative noise has an even smaller effect. Across all \((\sigma,b)\) combinations, accuracies remain in a narrow band around \(0.766\), with variations on the order of \(10^{-3}\). Because multiplicative scaling occurs before sign binarization, it tends not to flip hypervector components, leaving the decision boundaries almost unchanged. We therefore omit multiplicative plots for brevity and use them primarily as a sanity check confirming that additive perturbations—not gain variation—are the dominant accuracy-relevant non-idealities in this model.
Evaluation Methodology¶
Our methodology is designed to isolate intrinsic trends in accuracy, noise tolerance, and precision scaling while suppressing incidental variability. The multi-projection (three seeds) and multi-noise (ten draws) averaging ensures that each reported accuracy reflects stable behavior rather than idiosyncratic sampling effects. Sweeping the full grid of noise levels and bit-depths allows fine-grained trends to emerge, and per-dimension quantization statistics more faithfully capture the heterogeneous scales of projected dimensions in high-dimensional spaces. Together, these design choices yield smooth and interpretable accuracy–noise–precision curves and ensure that the observed trends are reproducible across seeds, projections, and quantization choices.
Summary of Findings¶
Taken together, our results show that HDC classifiers maintain remarkably stable accuracy across both noise and precision variations on MNIST: accuracies for 3–8 ADC bits differ by at most a few thousandths, even under additive noise as large as \(\sigma=0.20\). In the default energy regime, this robustness enables moderate (approximately \(10\%\)) energy savings when reducing precision from 8 to 4 bits. In the ADC-dominated regime, the same stability yields substantially larger savings—on the order of \(65\)–\(70\%\)—with virtually no degradation in classification accuracy. These findings demonstrate that HDC’s distributed, sign-based representations translate directly into practical opportunities for precision scaling in CiM accelerators, and they establish a reproducible quantitative map for choosing energy-efficient operating points. In practical terms, our results suggest that 4–5 ADC bits are “enough” for HDC on CiM for MNIST: they preserve baseline accuracy while substantially reducing modeled energy.
Discussion and Future Work¶
This study set out to quantify how analog noise and ADC precision affect the accuracy and energy efficiency of HDC when executed on CiM hardware. The preceding results provide a clear and quantitative answer to that question and establish a quantitative link between CiM-level precision parameters and HDC inference performance, bridging a gap that prior work has largely characterized only qualitatively356. Where prior CiM work has compared analog-in-memory accelerators against conventional digital baselines, our study instead focuses on intra-CiM precision scaling: the \(10–70\%\) energy savings we report are relative to an 8-bit CiM baseline, not to conventional digital designs. In doing so, the project demonstrates that HDC’s algorithmic robustness can translate directly into substantial circuit-level energy gains without retraining or architectural modification.
Beyond confirming HDC’s inherent noise tolerance, the findings have broader implications for analog hardware design. Our results reveal an important representation–compute interaction: HDC’s sign-based, high-dimensional representation absorbs analog variation, enabling CiM hardware to operate safely at lower precision and reclaim converter energy. Converter precision—traditionally fixed at design time—can instead be treated as a tunable resource. A CiM compiler or runtime controller could dynamically adjust ADC bit-depth in response to workload characteristics, quality-of-service constraints, or energy budgets, exploiting the wide plateau of stable accuracy revealed by our experiments. Such dynamic precision scaling aligns with a growing trend toward adaptive hardware–software co-design in sustainable machine learning systems, where robustness in the algorithmic layer enables flexibility and efficiency in the hardware layer.
Despite these promising outcomes, several limitations constrain the present simulator. The noise models capture independent Gaussian perturbations, but do not yet incorporate spatially correlated effects such as row-dependent IR-drop, device mismatch patterns, or long-term drift common in resistive memory technologies. The energy model, while calibrated to realistic orders of magnitude, remains parametric rather than derived from silicon measurements. In addition, although the transition to MNIST provides a more informative and widely used benchmark than earlier synthetic or low-dimensional datasets, our evaluation still does not cover higher-resolution image tasks, temporal data, or domains where feature distributions are more complex. These simplifications were deliberate to preserve interpretability, isolate first-order trends, and ensure reproducibility across seeds and projections. Nonetheless, they leave open important avenues for extending the fidelity of the model.
Future work will therefore extend the simulator along several complementary directions. First, device-aware modeling: incorporating empirical measurements of analog variability, retention loss, converter asymmetry, and spatially structured noise will enable more realistic mappings from circuit non-idealities to algorithmic accuracy. Second, system-level co-design: coupling the simulator with an analog-aware compiler or scheduling system would allow end-to-end exploration of precision scaling policies, ADC sharing strategies across CiM tiles, and workload-dependent adaptations to energy constraints. Third, architectural extensions: simulating associative memory operations, in-memory similarity search, or hybrid analog–digital pipelines would further illuminate how HDC’s properties interact with emerging CiM architectures and could remove the need for digitization at intermediate stages. Together, these efforts will refine both the accuracy and the applicability of the simulator, evolving it from a proof-of-concept study into a practical design tool for energy-efficient analog computing.
Ultimately, the intersection of HDC and CiM is not merely an exercise in modeling precision and noise—it represents a broader opportunity to align computational abstractions with physical constraints. By quantifying how algorithmic tolerance to analog noise and reduced precision translates into tangible energy savings, this work takes an early but essential step toward a unified framework in which analog efficiency and machine learning robustness co-evolve. Such alignment will be key to scaling intelligent computation sustainably in the decades ahead.
References¶
-
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016). ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ISCA, 14–26. https://doi.org/10.1145/3007787.3001139 ↩↩
-
Ankit, A., El Hajj, I., Chalamalasetti, S. R., Ndu, G., Foltin, M., Williams, R. S., Faraboschi, P., Hwu, W.-M. W., Strachan, J. P., Roy, K., & Milojicic, D. S. (2019). PUMA: A programmable ultra-efficient memristor-based accelerator for machine learning inference. ASPLOS, 715–731. https://doi.org/10.1145/3297858.3304049 ↩↩
-
Haensch, W., Raghunathan, A., Roy, K., Chakrabarti, B., Phatak, C. M., Wang, C.-C., & Guha, S. (2023). Compute-in-memory with non-volatile elements for neural networks: A review from a co-design perspective. Advanced Materials, 35(37), e2204944. https://doi.org/10.1002/adma.202204944 ↩↩↩↩
-
Xu, J., Liu, H., Duan, Z., Liao, X., Jin, H., Yang, X., Li, H., Liu, C., Mao, F., & Zhang, Y. (2024). ReHarvest: An ADC resource-harvesting crossbar architecture for ReRAM-based DNN accelerators. ACM Transactions on Architecture and Code Optimization (TACO), 21(3), Article 63, 1–26. https://doi.org/10.1145/3659208 ↩↩
-
Kleyko, D., Davies, M., Frady, E. P., Kanerva, P., Kent, S. J., Olshausen, B. A., Osipov, E., Rabaey, J. M., Rachkovskij, D. A., Rahimi, A., & Sommer, F. T. (2022). Vector symbolic architectures as a computing framework for emerging hardware. Proceedings of the IEEE, 110(10), 1538–1571. https://doi.org/10.1109/JPROC.2022.3209104 ↩↩↩
-
Karunaratne, G., Le Gallo, M., Cherubini, G., Benini, L., Rahimi, A., & Sebastian, A. (2020). In-memory hyperdimensional computing. Nature Electronics, 3, 327–337. https://doi.org/10.1038/s41928-020-0410-3 ↩↩↩
-
Kleyko, D., Rachkovskij, D. A., Osipov, E., & Rahimi, A. (2023). A survey on hyperdimensional computing aka vector symbolic architectures, part II: Applications, cognitive models, and challenges. ACM Computing Surveys, 55(9), Article 175, 1–52. https://doi.org/10.1145/3558000 ↩
-
Soliman, T., Chatterjee, S., Laleni, N., Müller, F., Kirchner, T., Wehn, N., Kämpfe, T., Chauhan, Y. S., & Amrouch, H. (2023). First demonstration of in-memory computing crossbar using multi-level cell FeFET. Nature Communications, 14, 6348. https://doi.org/10.1038/s41467-023-42110-y ↩