pytorch:激活函数、损失函数小结

总结pytorch里的激活函数、损失函数以及卷积、池化等

1.Activation function

Lineaer:

nn.Linear(in_features, out_features, bias=True)
    y=Ax+b
nn.Bilinear(in1_features, in2_features, out_features, bias=True)
    y=x1*A*x2+b

Non-linear

nn.ReLU(inplace=False)
nn.ReLU6(inplace=False)
nn.ELU(alpha=1.0, inplace=False)
nn.SELU(inplace=False)
nn.PReLU(num_parameters=1, init=0.25)
nn.LeakyReLU(negative_slope=0.01, inplace=False)
nn.Threshold(threshold, value, inplace=False)
nn.Hardtanh(min_val=-1, max_val=1, inplace=False, min_value=None, max_value=None)
nn.Sigmoid
nn.LogSigmoid
nn.Tanh
nn.Tanhshrink
nn.Softplus(beta=1, threshold=20)
nn.Softmax(dim=None)
nn.LogSoftmax(dim=None)
nn.Softmax2d
nn.Softmin(dim=None)
nn.Softshrink(lambd=0.5)
nn.Softsign

nn.xx与nn.functional.xx区别:

前者时包装好的类，后者是可直接调用的函数；nn.xx类的forward函数调用了nn.functional.xx函数

2.Loss function

nn.L1Loss(size_average=True, reduce=True)
nn.MSELoss(size_average=True, reduce=True)
nn.CrossEntropyLoss(weight=None, size_average=True, ignore_index=-100, reduce=True)
nn.NLLLoss(weight=None, size_average=True, ignore_index=-100, reduce=True)
nn.PoissonNLLLoss(log_input=True, full=False, size_average=True, eps=1e-08)
nn.NLLLoss2d(weight=None, size_average=True, ignore_index=-100, reduce=True)
nn.KLDivLoss(size_average=True, reduce=True)
nn.BCELoss(weight=None, size_average=True)
nn.BCEWithLogitsLoss(weight=None, size_average=True)
nn.MarginRankingLoss(margin=0, size_average=True)
nn.HingeEmbeddingLoss(margin=1.0, size_average=True)
nn.MultiLabelMarginLoss(size_average=True)
nn.SmoothL1Loss(size_average=True, reduce=True)
nn.SoftMarginLoss(size_average=True)
nn.CosineEmbeddingLoss(margin=0, size_average=True)

3.Normalization

nn.BatchNorm1d(num_features, eps=1e-05, momentum=0.1, affine=True)
nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True)
nn.BatchNorm3d(num_features, eps=1e-05, momentum=0.1, affine=True)
nn.InstanceNorm1d(num_features, eps=1e-05, momentum=0.1, affine=False)
nn.InstanceNorm2d(num_features, eps=1e-05, momentum=0.1, affine=False)
nn.InstanceNorm3d(num_features, eps=1e-05, momentum=0.1, affine=False)

4.Convolution

Conv1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
nn.Conv3d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
nn.ConvTranspose1d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1)[s
nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1)
nn.ConvTranspose3d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1)

5.Pooling

nn.MaxPl1d(knl_iz, tid=Nn, padding=0, dilatin=1, tn_indi=Fal, il_md=Fal)
nn.MaxPl2d(knl_iz, tid=Nn, padding=0, dilatin=1, tn_indi=Fal, il_md=Fal)
nn.MaxPl3d(knl_iz, tid=Nn, padding=0, dilatin=1, tn_indi=Fal, il_md=Fal)
nn.Maxnpl1d(knl_iz, tid=Nn, padding=0)
nn.Maxnpl2d(knl_iz, tid=Nn, padding=0)
nn.Maxnpl3d(knl_iz, tid=Nn, padding=0)
nn.AvgPl1d(knl_iz, tid=Nn, padding=0, il_md=Fal, nt_inld_pad=T)
nn.AvgPl2d(knl_iz, tid=Nn, padding=0, il_md=Fal, nt_inld_pad=T)
nn.AvgPl3d(knl_iz, tid=Nn, padding=0, il_md=Fal, nt_inld_pad=T)
nn.FatinalMaxPl2d(knl_iz, tpt_iz=Nn, tpt_ati=Nn, tn_indi=Fal, _andm_ampl=Nn)
nn.LPPl2d(nm_typ, knl_iz, tid=Nn, il_md=Fal)
nn.AdaptivMaxPl1d(tpt_iz, tn_indi=Fal)
nn.AdaptivMaxPl2d(tpt_iz, tn_indi=Fal)
nn.AdaptivMaxPl3d(tpt_iz, tn_indi=Fal)
nn.AdaptivAvgPl1d(tpt_iz)
nn.AdaptivAvgPl2d(tpt_iz)
nn.AdaptivAvgPl3d(tpt_iz)

6.Dropout

nn.Dropout(p=0.5, inplace=False)
nn.Dropout2d(p=0.5, inplace=False)
nn.Dropout3d(p=0.5, inplace=False)
nn.AlphaDropout(p=0.5)

7.Distance functions

nn.CosineSimilarity(dim=1, eps=1e-08)
nn.PairwiseDistance(p=2, eps=1e-06)

赏

谢谢你请我吃糖果

支付宝

微信