http://hdl.handle.net/10174/38442 2026-04-06T13:04:53Z http://hdl.handle.net/10174/41453 Title: Performance Evaluation of NLP Models for European Portuguese: Multi-GPU/Multi-node Configurations and Optimization Techniques Authors: Santos, Daniel; Miquelina, Nuno; Schmidt, Daniela; Quaresma, Paulo; Nogueira, Vítor Beires Abstract: Natural Language Processing (NLP) research has predominantly focused on the English language, leading to a wealth of resources and advancements tailored to English. However, there is a growing need to extend these capabilities to other languages, such as European Portuguese, to ensure the inclusivity and accessibility of NLP technologies. In this study, we explore the evaluation of NLP models in the European Portuguese language using a multi-GPU/multi-node machine. We utilized various tools such as PyTorch, Accelerate, Transformers, and DeepSpeed with ZeRO Stage 3 to handle the computational demands of large-scale model training. We provide all the key aspects of our methodology to evaluate various models on translated GLUE tasks. Additionally, we introduce AiBERTa, a base model with 110 million parameters, developed and pre-trained on a corpus tailored for European Portuguese. This research highlights the effectiveness of advanced tools and distributed computing in scaling NLP model training, providing a foundation for future enhancements in European Portuguese language processing. 2025-02-17T00:00:00Z http://hdl.handle.net/10174/41452 Title: A Galician-Portuguese Generative Model Authors: Gamallo, Pablo; Rodríguez, Pablo; Sotelo, Susana; Miquelina, Nuno; Paniagua, Silvia; Schmidt, Daniela; de-Dios-Flores, Iria; Quaresma, Paulo; Bardanca, Daniel; Pichel, José Ramom; Nogueira, Vítor; Barro, Senén Abstract: Large language models (LLMs) have revolutionized natural language processing, but their predominant focus on English has resulted in biases and performance differences across various languages. This situation is maintained in generative multilingual models, where English continues to be the predominant language. In these models, the presence of European Portuguese is marginal and that of the Galician variety is almost residual. In this work, we describe an open-source Galician-Portuguese generative model, Carvalho_pt-gl, focused precisely on these two language variants, which are very close lexically and syntactically. The model was trained using a GPT architecture with 1.3 billion parameters on more than 6B words, balanced between the two varieties. The strategy of continual pertaining was used to adapt a pre-existing LLM that was trained on a trilingual dataset with related languages, thereby overcoming the data limitations that would be faced if the training was started from scratch. Evaluation results involving task-based datasets from standardized benchmarks indicate a promising performance. These findings highlight the critical importance of supporting linguistic diversity in generative models. 2024-11-16T00:00:00Z http://hdl.handle.net/10174/41401 Title: Parameter Efficient Fine-Tunning of LLMs: Application to Machine Translation from English to Portuguese Authors: Santos, Daniel; Nogueira, Vitor; Quaresma, Paulo Abstract: Fine-tuning Large Language Models (LLMs) for specific tasks, such as machine translation, is a computationally expensive process that often requires substantial hardware resources. Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA), offer a resource-efficient alternative by significantly reducing the number of trainable parameters and memory requirements. In this work, we compare the performance and memory efficiency of LoRA and QLoRA on English-Portuguese translation tasks, utilizing two cutting edge LLMs, Meta LLaMA 3.1 8B and Mistral 7B. Our experiments demonstrate that both LoRA and QLoRA achieve substantial memory savings. Moreover, this work underscores the practical advantages of LoRA and QLoRA in resource-constrained environments, providing a foundation for further optimization and experimentation in machine translation using large language models. 2025-01-01T00:00:00Z http://hdl.handle.net/10174/41362 Title: Intelligent Data Engineering and Automated Learning – IDEAL 2024 Authors: Nogueira, Vitor Editors: Julian, Vicente; Camacho, David; Yin, Hujun; Nogueira, Vitor; Novais, Paulo; Tallón-Ballesteros, Antonio Abstract: This two-volume set, LNCS 15346 and LNCS 15347, constitutes the proceedings of the 25th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2024, held in Valencia, Spain, during November 20–22, 2024. The 86 full papers and 6 short papers presented in this book were carefully reviewed and selected from 130 submissions. IDEAL 2024 is focusing on Big Data Analytics and Privacy, Machine Learning & Deep Learning for Real-World Applications, Data Mining and Pattern Recognition, Information Retrieval and Management, Bio and Neuro-Informatics, and Hybrid Intelligent Systems and Agents. 2024-11-01T00:00:00Z