• Search for:

• Recent Posts

  • Secretary of the ITACA Insitute
  • Participation in EDCC 2025
  • Paper available at ACM
  • Paper accepted at LADC 2024
  • Paper available at Springer Link

• Recent Comments

• Archives

  • March 2025
  • February 2025
  • December 2024
  • October 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • January 2024
  • September 2023
  • July 2023
  • June 2023
  • March 2023
  • February 2023
  • November 2022
  • October 2022
  • September 2022
  • July 2022
  • June 2022
  • May 2022
  • October 2021
  • September 2021
  • June 2021

Our colleague David de Andrés has been appointed as administration manager of the ITACA Institute at Universitat Politècnica de València.

Congratulations!

During next EDCC (20th European Dependable Computing Conference), that will be held in Lisbon (Portugal) in April 2025, the GSTF research group will present three extended fast-abstracts, showing our current research lines.
These works are:

Title: Towards a Novel 8-bit Floating-point Format to Increase Robustness in Convolutional Neural Networks

Authors: Luis-J. Saiz Adalid, Juan-Carlos Ruiz-García, Joaquín Gracia-Morán, David de Andrés, J.-Carlos Baraza-Calvo, Daniel Gil-Tomás, Pedro Gil-Vicente

Abstract: Convolutional Neural Networks (CNNs) are widely adopted in Artificial Intelligence applications, particularly in computer vision and other deep learning tasks. Their performance relies on millions of parameters, including weights and biases, which are optimized during training, stored, and utilized during inference. Traditionally, these parameters are represented using the 32-bit IEEE-754 single-precision floating-point format. However, research has shown that excess precision in this format is not always required to maintain accuracy, motivating the adoption of reduced-precision 16-bit formats. A natural progression of this trend is representing real numbers using 8-bit formats. However, existing proposals often suffer from precision loss, negatively impacting CNN accuracy. In this extended abstract, we propose a novel 8-bit floating-point format designed to enhance reliability in CNNs, due to its reduced memory footprint, enough precision, and well-fitted range. We evaluate its advantages and limitations through comparative analysis. Initial findings suggest that our format improves computational efficiency while preserving accuracy comparable to 32-bit networks, increasing reliability. However, further experimental validation is necessary.

Title: Towards SW-based Robustness Assessment of HW Accelerators for Quantized CNNs

Authors: Juan Carlos Ruiz, David de Andrés, Juan-Carlos Baraza-Calvo, Luis-José Saiz-Adalid,
Joaquín Gracia-Morán, Daniel Gil-Tomás, Pedro Gil-Vicente

Abstract: Quantized Convolutional Neural Networks (QCNNs) are widely adopted in resource-limited environments due to their reduced memory footprint, lower power consumption, and faster execution. Hardware (HW) accelerators further enhance these benefits by optimizing QCNN execution for available resources, a crucial aspect in embedded systems. Typically, the development cycle of neural networks begins with a software (SW) model, which is later refined and implemented in HW. Leveraging this initial SW model for early robustness assessments is an attractive approach, as it allows fault tolerance evaluation before committing to HW implementation. However, for such assessments to be meaningful, they must accurately reflect the fault behavior that would occur in HW. This paper examines the limitations of naive fault injection approaches in SW-based QCNN models and demonstrates how they can lead to misleading conclusions about HW robustness. The key issue arises from the internal multi-component representation of quantized parameters, which differs from direct floating-point storage. Our analysis highlights that simplistic bit-flip injections in SW-based QCNNs do not necessarily translate to equivalent faults in their HW implementations, leading to inaccuracies in robustness evaluation. While we do not propose a specific fault injection methodology, we identify critical challenges that must be addressed to ensure that early-stage SW evaluations on QCNNs have potential to provide valid, cost-effective insights into the resilience of their HW implementations.

Title: Initial insights into synthesis overheads caused by C-based Error Correction Codes implementations

Authors: Joaquín Gracia-Morán, David de Andrés, Luis-J. Saiz-Adalid, Juan Carlos Ruiz, J.-Carlos Baraza-Calvo, Daniel Gil-Tomás, Pedro J. Gil-Vicente

Abstract: Error Correction Codes (ECCs) are increasingly used in safety-critical systems, such as hardware accelerators for cryptographic computations and neural network inference. These systems require high reliability, making ECCs essential for mitigating soft errors and improving fault tolerance. Thus, the demand for efficient ECC implementations is rising, necessitating faster design and deployment processes. Traditional hardware design approaches, such as Register-Transfer Level (RTL) development, can be time-consuming and very complex. High-Level Synthesis (HLS) enables the automatic transformation of C-based ECC models into hardware descriptions, reducing development effort while allowing design-space exploration. This methodology facilitates rapid prototyping and optimization, enabling the evaluation of different architectural choices without manually modifying the RTL code. However, coding styles, algorithmic transformations, and optimization strategies in C-based can directly affect the synthesized hardware’s performance metrics, including area utilization, power consumption, and latency.
This work provides initial insights into how different C-based ECC design choices influence the final hardware implementation. To do this, we have analyzed synthesis results under various ECC configurations.

The paper entitled “Allocating ECC parity bits into BF16-encoded CNN parameters: A practical experience report” written by Joaquín Gracia-Morán, Juan Carlos Ruiz Garcia, David de Andrés Martínez and Luis Jose Saiz-Adalid is available at ACM: https://dl.acm.org/doi/10.1145/3697090.3697092

Abstract

Using low-precision data types, like the Brain Floating Point 16 (BF16) format, can reduce Convolutional Neural Networks (CNNs) memory usage in edge devices without significantly affecting their accuracy. Adding in-parameter zero-space Error Correction Codes (ECCs) can enhance the robustness of BF16-based CNNs. However, implementing this technique raises practical questions. For instance, when the available invariant1 and non-significant2 bits in parameters for error correction are sufficient for the required protection level, the proper selection and combination of these bits become crucial. On the other hand, if the set of available bits is inadequate, converting nearly invariant bits to invariants might be considered. These decisions impact ECC decoder complexity and may affect the overall CNN performance. This report examines such implications using Lenet-5 and GoogLenet as case studies.

The paper entitled “Allocating ECC parity bits into BF16-encoded CNN parameters: A practical experience report” written by Joaquín Gracia-Morán, Juan Carlos Ruiz Garcia, David de Andrés Martínez and Luis Jose Saiz-Adalid has been accepted at 13th Latin-American Symposium on Dependable and Secure Computing (LADC 2024), that will be held in Recife (Brazil) next November.

Abstract

Using low-precision data types, like the Brain Floating Point 16 (BF16) format, can reduce Convolutional Neural Networks (CNNs) memory usage in edge devices without significantly affecting their accuracy. Adding in-parameter zero-space Error Correction Codes (ECCs) can enhance the robustness of BF16-based CNNs. However, implementing this technique raises practical questions. For instance, when the available invariant1 and non-significant2 bits in parameters for error correction are sufficient for the required protection level, the proper selection and combination of these bits become crucial. On the other hand, if the set of available bits is inadequate, converting nearly invariant bits to invariants might be considered. These decisions impact ECC decoder complexity and may affect the overall CNN performance. This report examines such implications using Lenet-5 and GoogLenet as case studies.

The paper entitled “In-Memory Zero-Space Floating-Point-based CNN protection using non-significant and invariant bits”, and written by Juan Carlos Ruiz Garcia, Luis Jose Saiz-Adalid, David de Andrés Martínez, and Joaquín Gracia-Morán, published at SAFECOMP 2024, can be accesed at this link.

Abstract

Convolutional Neural Networks (CNNs) have accomplished significant success in various domains, including transportation, health care and banking. Millions of weights, loaded from main memory into the internal buffers of CNN accelerators, are repeatedly used in the inference process. Accidental and malicious bit-flips targeting these buffers may negatively impact the CNN’s accuracy. This paper proposes a methodology to tolerate the effect of (multiple) bit-flips on floating-point-based CNNs using the non-significant and the invariant bits of CNN parameters. The former, determined after fault injection, do not significantly affect the accuracy of the inference process regardless of their value. The latter, determined after analyzing the network parameters, have the same value for all of them. Slight modifications can be applied to carefully selected parameters to increase the number of invariant bits. Since non-significant and invariant bits do not require protection against faults, they are employed to store the parity bits of error control codes. The methodology preserves the CNN accuracy, keeps its memory footprint, and does not require any retraining. Its usefulness is exemplished through the FP32 and BFloat16 versions of the LeNet-5 and GoogleNet CNNs.

This year, we are celebrating the 25th anniversary of Instituto ITACA.

Congratulations!!

Juan C. Ruiz has presented the research work done by the Fault tolerant Systems research group of UPV at 86th IFIP WG 10.4 Meeting at Gold Coast, Australia, entitled “On improving the robustness of convolutional neural networks”.

Juan C. Ruiz, Luis J. Saiz and Joaquín Gracia have attended the CEDI 2024, that was held in A Coruña, Spain. During these days, they have contacted with others researches, exchanging ideas and establishing possible future collaborations

Juan C. Ruiz is attending the IFIP WG 10.4 meeting in Gold Coast, Australia. The IFIP WG 10.4 is aimed at identifying and integrating approaches, methods and techniques for specifying, designing, building, assessing, validating, operating and maintaining computer systems which should exhibit some or all of these attributes.

This June 24, the VERDI workshop (co-located with DSN 2024) was held in Brisbane, Australia. In this workshop, Juan C. Ruiz has been the invited speaker with a talk entitled “On improving the robustness of convolutional neural networks using in-parameter zero-space error correction codes“.

The program of VERDI 2024 can be seen here.

This talk has been sponsored by the Agencia Estatal de Investigación (Ministerio de Ciencia e Innovación) through the DEFADAS project.

• Categories

  • Congress
  • Congress Paper
  • DEFADAS
  • General News
  • Journal Paper
  • Keynote
  • Meeting
  • PIME-UPV
  • Research
  • Teaching