Current Issue

The classification of skin cancer poses significant challenges due to the presence of artifacts, variations in image resolution, and subtle differences in features between various cancer types. Clinical procedures for examining skin lesions are often perceived by patients as complex and uncomfortable and can fall short of effectively differentiating between lesion categories. Advanced techniques in Machine Learning and Computer Vision present a promising avenue to address the hurdles associated with skin lesion classification. The foundational step in Computer-Aided Diagnosis(CAD) systems involves generating discriminative features to categorize lesions as normal or pathological. The classification of skin cancer presents considerable challenges due to the presence of artifacts, varying image resolutions, and subtle distinguishing features among different cancer types. Such complexities render clinical procedures for skin lesion assessment both intricate and oftentimes ineffectual in accurately differentiating between lesion categories.

In response to these diagnostic challenges, we have crafted a novel model based on the DenseNet architecture[14]. Over the past several years, significant strides have been made to support medical professionals in the accurate recognition of skin lesions. The development of CAD systems for skin abnormalities has become a rapidly evolving area of research. When it comes to the categorization of images, there exist two predominant techniques: conventional machine learning methods and the more recent advancements in deep learning.

Fig. 1 shows that the proposed model consists of dense blocks from the image classification model DenseNet. The model includes three dense blocks and attention mechanisms, respectively. Dense blocks are distinguished by their dense connectivity patterns among layers within the block. In contrast to conventional architectures with sequential layer connections, each layer in a dense block receives direct input from all preceding layers.

Fig. 1. 
Modified 6-layer dense block with growth rate of k=4 of the proposed model

This dense connectivity promotes feature reuse, facilitates gradient flow during training, and enhances the overall effectiveness of the network. This choice is motivated by the fact that the bottleneck section of the DenseNet utilizes an input feature size of 64, while the initial input image possesses a shape of 3x32x32 with 3 input features.

Fig. 1 illustrates the modified dense block, emphasizing enhanced connectivity and efficient feature reuse within the network. The diagram uses different colors to represent the components: gray for the input image (CxHxW), blue for convolution layers, yellow for ReLU activation functions, and green for 3D Batch Normalization. This visual representation clarifies the data flow and interactions between layers, aiding in the understanding of the model's structure and performance improvements.

The utilization of a convolution layer allows an increase in the number of input features by 64, addressing the potential inadequacy of input features if only the original 3 channels were used. The input image, denoted as traverses through a sequence of l dense and attention blocks, generating outputs at each block for the l-th block in the sequence.

where [x_0,x_1⋯,x_(l-1),] denotes the amalgamation of feature-maps generated in layers [0, . . . , −1]. The term DenseNet is attributed to this network architecture due to its dense connectivity. To facilitate implementation, we streamline the multiple inputs of H_l in equation (1) by combining them into a unified tensor. If each function H_l generates k feature-maps, then it can be deduced that the -th layer encompasses k_0+ k × (l−1) input feature-maps, where k_0 represents the number of channels in the input layer.

Here, the hyperparameter k is denoted as the growth rate of the network, in our proposed model we use k = 4. The feature maps essentially represent the global state of the network, and each layer augments this state by adding k feature maps. The growth rate serves as a regulatory factor, determining the extent to which each layer contributes new information to the global state. Though each layer typically yields only k output feature maps, it frequently accommodates a more extensive array of inputs. It has been observed in prior studies that the incorporation of a 1×1 convolution as a bottleneck layer preceding each 3×3 convolution proves effective in diminishing the number of input feature maps, thereby enhancing computational efficiency.

The bottleneck layer is represented as BN-ReLU-Conv (1×1) BN-ReLU-Conv (3×3) in version H_l. In our experimental setups, we assign each 1×1 convolution the task of producing growth rate * k feature maps.

In the realm of CNNs, attention modules play a crucial role in enhancing learning by directing the model to concentrate on pertinent information while disregarding irrelevant background details. In the context of object detection, the salient data comprises the cropped representation of objects or target classes that necessitate classification and localization within an image. The Convolutional Block Attention Module (CBAM) segregates into dual submodules: channel attention and spatial attention. Each submodule incorporates convolutional strata complemented by maximal and average pooling procedures.

The input feature F, a product of the preceding modified dense block, undergoes a fusion of Max and Avg poolings, yielding two distinct spatial context descriptors, Favg c and Fmaxc. Subsequently, these descriptors are propagated through a Multi-layer Perceptron(MLP) equipped with a singular hidden layer, culminating in a feature representation McϵRcx1x1.

To curtail parameter proliferation, the MLP condenses its siRcrx1x1 per output feature, with r denoting the reduction ratio. Following the application of the shared network to each descriptor, the resultant feature vectors are amalgamated via an element-wise additive process.

In this study, we preserve the integrity of the Channel Attention Block(ChaB), noting its previously established efficacy, and introduce only Maxpooling to diminish the vector map dimensions and enhance informational retrieval.

Fig. 2 the CBAM is fed an intermediate feature map FMDB∈RCxHxW, emanating from a Modified Dense Block (MDB), methodically deduces two forms of attention maps. The first is a 1D channel attention Mc∈RCxHxW and the second is a 2D spatial attention map, Ms∈RCxHxW, as delineated in Equation (2).

Fig. 2. 
Light-modified channel attention block

Given the emphasis on the classification task, the channel attention module, which is adept at discerning 'what' aspects are present within the input feature, is utilized. Consequently, only M_c(F) is extracted and integrated with the output of the i-th MDB as per Equation (3). This selective application ensures that the focus remains on channel-wise feature relevance, pertinent to identifying class-discriminative information.

The intire process of Light-modified ChaB can be encapsulated by Equation (4) where σ represents the sigmoid activation function. The weights of the i-th layer of the MLP, denoted as WiϵRcrxc, and the weights of the preceding layer, WiϵRcxcr , are specifically designed to be shared across both inputs, facilitating a consistent parameterization for the dual pathways involving MaxPooling and AveragePooling. Following this parameter sharing and the application of the ReLU activation function, a MaxPooling operation is employed as delineated in Equation (5). This incorporation of MaxPooling constitutes a subtle yet impactful modification aimed at optimizing the feature selection process within the channel attention framework, enhancing the module's efficiency in focusing on pivotal aspects pertinent to the classification task at hand.

Combine the outcomes of average pooling and max pooling and the extra maxpoling, we transmit the concatenated result through a compact convolutional block featuring a 3x3 kernel size. As a consequence, the output of Light-modified ChaB proceeds to theto the next i-th MDB. The dense block and attention block feature extraction cycle is repeated about three times, then the output goes into flatten layer and the final output comes from activation function softmax, that takes as input a vector of numerical scores or logits and normalizes them into a probability distribution, as shown:

where z_i input vector of each element, by raising e to the power of the input vector and dividing the result by the sum of the exponentiated values across all elements in the vector.

The International Skin Imaging Collaboration (ISIC) 2018 dataset[20] represents a pivotal resource in the domain of dermatological image analysis, specifically focusing on the automated detection and classification of skin lesions. Assembled for the ISIC Skin Lesion Analysis Towards Melanoma Detection Challenge, this comprehensive dataset encompasses a total of 37,649 dermatoscopic images.

Table 1 presents a comparative analysis of various classification networks when evaluate on the ISIC dataset. The networks compare include DenseNet121, VGG19, MobileNet, EfficientNetB0, ResNet50, and a the proposed model. For DenseNet121, the training accuracy is recorded at 96.56%, with a corresponding validation accuracy of 95.70%. It shows a training loss of 0.073 and a validation loss of 0.124 over 60 epochs, leading to a test accuracy of 95.94%. The model has 8 million parameters.

Furthermore, the performance of the VGG19 shows a training accuracy of about 91.23% and a validation accuracy of 88.56%, with higher loss values of 0.279 for training and 0.342 for validation. Despite 60 epochs of training, the test accuracy is lower than DenseNet121, at 90.27%, and the model has a significantly larger number of parameters, amounting to 21 million.

MobileNet achieves a training accuracy of 96.87% and a validation accuracy of 94.34%, with training and validation losses of 0.063 and 0.145, respectively. After 60 epochs, the test accuracy is 93.43%, and the model Is relatively efficient with 4 million parameters. EfficientNetB0 reports training and validation accuracies of 96.78% and 94.67%, respectively, with the lowest training loss of 0.025 and a validation loss of 0.213.

EfficientNetB0, the model with the least parameters at 6 million, leads with the highest training and validation accuracies of 96.78% and 95.14%, respectively.

ResNet50, with 25.6 million parameters, has training and validation accuracies of 89.84% and 75.15%. ResNet50 shows a training accuracy of 87.44% and a validation accuracy of 86.26%, with higher losses (0.321 for training and 0.632 for validation).