publications
publications by categories in reversed chronological order. generated by jekyll-scholar.
2024
- Revisiting LARS for Large Batch Training Generalization of Neural NetworksKhoi Do, Duong Nguyen, Hoa Nguyen, Long Tran-Thanh, Nguyen-Hoang Tran, and 1 more author2024
This paper explores Large Batch Training techniques using layer-wise adaptive scaling ratio (LARS) across diverse settings, uncovering insights. LARS algorithms with warm-up tend to be trapped in sharp minimizers early on due to redundant ratio scaling. Additionally, a fixed steep decline in the latter phase restricts deep neural networks from effectively navigating early-phase sharp minimizers. Building on these findings, we propose Time Varying LARS (TVLARS), a novel algorithm that replaces warm-up with a configurable sigmoid-like function for robust training in the initial phase. TVLARS promotes gradient exploration early on, surpassing sharp optimizers and gradually transitioning to LARS for robustness in later phases. Extensive experiments demonstrate that TVLARS consistently outperforms LARS and LAMB in most cases, with up to 2% improvement in classification scenarios. Notably, in all self-supervised learning cases, TVLARS dominates LARS and LAMB with performance improvements of up to 10%.
- Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting GradientsMinh Duong Nguyen, Khanh Le, Khoi Do, Nguyen H. Tran, Duc Nguyen, and 2 more authors2024
In personalized Federated Learning (pFL), high data heterogeneity can cause significant gradient divergence across devices, adversely affecting the learning process. This divergence, especially when gradients from different users form an obtuse angle during aggregation, can negate progress, leading to severe weight and gradient update degradation. To address this issue, we introduce a new approach to pFL design, namely Federated Learning with Layer-wise Aggregation via Gradient Analysis (FedLAG), utilizing the concept of gradient conflict at the layer level. Specifically, when layer-wise gradients of different clients form acute angles, those gradients align in the same direction, enabling updates across different clients toward identifying client-invariant features. Conversely, when layer-wise gradient pairs make create obtuse angles, the layers tend to focus on client-specific tasks. In hindsights, FedLAG assigns layers for personalization based on the extent of layer-wise gradient conflicts. Specifically, layers with gradient conflicts are excluded from the global aggregation process. The theoretical evaluation demonstrates that when integrated into other pFL baselines, FedLAG enhances pFL performance by a certain margin. Therefore, our proposed method achieves superior convergence behavior compared with other baselines. Extensive experiments show that our FedLAG outperforms several state-of-the-art methods and can be easily incorporated with many existing methods to further enhance performance.
- Block-Based Texture Features for Chromoendoscopy ClassificationViet Dung Nguyen, Hoang Nam Trinh, and Hoang Khoi DoIn 9th International Conference on the Development of Biomedical Engineering in Vietnam, 2024
Globally, there has been an upsurge in concern about stomach cancer in recent years. One of the key factors contributing to gastrointestinal (GI) tract abnormalities, such as ulcers and inflammation, is an improper diet. Additionally, these anomalies might aid in the growth of stomach cancer. Compared to biopsy, endoscopy is a less invasive way to screen gastric cancer and other GI tract abnormalities. In chromoendoscopy, one of the advancements to endoscopy, abnormal regions are made more visible visually by spraying colors over the mucosal surface. However, identifying abnormal regions in the frames taken during a chromoendoscopic session is still challenging. Basic visual cues like textures help to identify objects. They are helpful for inspecting abnormal spots in endoscopic frames as well. Several methods for representing texture and classifying endoscopic images have been developed. In this study, we proposed block-based texture features, especially Block Difference of Inverse Probabilities (BIDP) and Block Variation of Local Correlation Coefficients (BVLC), to represent the chromoendoscopic texture. The extracted features are then used to classify the chromoendoscopic image into normal and abnormal. On the dataset obtained using an Olympus CV-180 endoscope at the Portuguese Institute of Oncology (IPO) Hospital in Porto, Portugal, the proposed scheme has a classification accuracy of 88.9% and an area under the curve (AUC) value of 0.95, according to experimental results. It is proved that combining the BDIP and BLVC texture descriptors exhibits an excellent discrimination performance on the chromoendoscopic images. Though the proposed method is limited to chromoendoscopic images, it has the potential to be generalized by using images obtained from various imaging modalities under a variety of unhealthy conditions.
2023
- Personal Federated Learning via Momentum Target with Self-ImprovementT-Binh Nguyen, H-Khoi Do, M-Duong Nguyen, and T-Hoa NguyenIn Intelligent Systems and Networks, 2023
Federated learning (FL) is a new artificial intelligence concept that enables Internet-of-Things (IoT) devices to learn a collaborative model without sending the raw data to centralized nodes for processing. Despite numerous advantages, Federated Learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, we propose a novel personalized federated learning method via momentum adaptation, the so-called pFLTI. Specifically, pFLTI generates the target model by adapting from the temporal ensemble of the meta-learner, i.e., the momentum network. This momentum network and its task-specific adaptations enjoy a favorable generalization performance, enabling self-improving of the meta-learner through knowlodels to find the across task relationsedge distillation. Moreover, we found that perturbing parameters of the meta-learner, e.g., dropout, further stabilize this self-improving process by preventing fast convergence of the distillation loss during meta-training. Our experimental results demonstrate that our algorithm is awesome.
2022
- Skin Lesion Classification on Imbalanced Data Using Deep Learning with Soft AttentionViet Dung Nguyen, Ngoc Dung Bui, and Hoang Khoi DoSensors, 2022
Today, the rapid development of industrial zones leads to an increased incidence of skin diseases because of polluted air. According to a report by the American Cancer Society, it is estimated that in 2022 there will be about 100,000 people suffering from skin cancer and more than 7600 of these people will not survive. In the context that doctors at provincial hospitals and health facilities are overloaded, doctors at lower levels lack experience, and having a tool to support doctors in the process of diagnosing skin diseases quickly and accurately is essential. Along with the strong development of artificial intelligence technologies, many solutions to support the diagnosis of skin diseases have been researched and developed. In this paper, a combination of one Deep Learning model (DenseNet, InceptionNet, ResNet, etc) with Soft-Attention, which unsupervisedly extract a heat map of main skin lesions. Furthermore, personal information including age and gender are also used. It is worth noting that a new loss function that takes into account the data imbalance is also proposed. Experimental results on data set HAM10000 show that using InceptionResNetV2 with Soft-Attention and the new loss function gives 90 percent accuracy, mean of precision, F1-score, recall, and AUC of 0.81, 0.81, 0.82, and 0.99, respectively. Besides, using MobileNetV3Large combined with Soft-Attention and the new loss function, even though the number of parameters is 11 times less and the number of hidden layers is 4 times less, it achieves an accuracy of 0.86 and 30 times faster diagnosis than InceptionResNetV2.