notebook 3, implementation task section, Conv layer

notebook 3, implementation task section, Conv layer

di Oleksandr Tarasov -
Numero di risposte: 1

Hi guys,

currently working on the Conv layer implementation and was a little bit confused by ConvLayer class in the "Implementation section":

class ConvolutionalLayer(BaseLayer):

def __init__(self, stride_shape, kernel_shape, n_kernels, learning_rate, weights_initializer=UniformRandom(), bias_initializer=Const(0.1)):
"""
param: stride: tuple in the form of (np, nq) which denote the subsampling factor of the
convolution operation in the spatial dimensions
param: kernel_shape: integer tuple in the form of (n_channels, m, n) where n_channels is
the number of input channels and m x n is the size of the filter kernels
param: n_kernels (int): number of kernels and therefore the number of output channels
param: learning_rate (float): learning rate of this layer
param: weights_initializer: initializer object for the filter weights
param: bias_initializer: initializer object for the bias
"""

The default value for the weights_initializer and bias_initializer are classes that are not referenced anywhere in the notebook. Could you please advise what should we do with this notation?

In riposta a Oleksandr Tarasov

Re: notebook 3, implementation task section, Conv layer

di George Ciubotariu -

class Initializer:
    """ Base class for initializers. """
    def initialize(self, weight_shape):
        """ Return weights initialized according to the subclass definition. 
            Required to work for arbitrary weight shapes.
            Base class. 
        """
        
        # Raises an exeption in base class.
        raise NotImplementedError('Method is not implemented')

        
class Const(Initializer):
    
    def __init__(self, value):
        """ Create a constant initializer.
            params: value (float): constant that is used for initialization of weights
        """
        # TODO: Implement
        pass

    def initialize(self, weight_shape):
        """ Return a new array of weights initialized with a constant value provided by self.value.
            param: weight_shape: shape of the new array
            returns (np.ndarray): array of the given shape
        """
        # TODO: Implement
        pass

class UniformRandom(Initializer):
    
    def initialize(self, weight_shape):
        """ Return a new array of weights initialized by drawing from a uniform distribution with range [0, 1].
            param: weight_shape: shape of new array
            returns (np.ndarray): array of the given shape
        """
        # TODO: Implement
        pass

I assume that is what you're interested in.
You may implement this or you could just use two simple functions for returning a numpy array filled up with the same value (for Const init), or values generated by calling one of the built-in numpy random functions (for Uniform init).
Please don't hesitate to ask further questions.