Activation

from chemprop.nn.utils import Activation

Activation functions

The following activation functions can be specified by name (e.g., activation = "relu"):

for activation in Activation:
    print(activation)

relu
leakyrelu
prelu
tanh
elu

from chemprop.nn import AtomMessagePassing

mp = AtomMessagePassing(activation="relu")

mp

AtomMessagePassing(
  (W_i): Linear(in_features=72, out_features=300, bias=False)
  (W_h): Linear(in_features=314, out_features=300, bias=False)
  (W_o): Linear(in_features=372, out_features=300, bias=True)
  (dropout): Dropout(p=0.0, inplace=False)
  (tau): ReLU()
  (V_d_transform): Identity()
  (graph_transform): Identity()
)

Custom

Custom activation functions can be used by passing a PyTorch module to the MessagePassing or Predictor constructor.

See Non-linear Activations (weighted sum, nonlinearity) and Non-linear Activations (other) for common PyTorch activation functions.

import torch

mp = AtomMessagePassing(activation=torch.nn.Softplus())

mp

AtomMessagePassing(
  (W_i): Linear(in_features=72, out_features=300, bias=False)
  (W_h): Linear(in_features=314, out_features=300, bias=False)
  (W_o): Linear(in_features=372, out_features=300, bias=True)
  (dropout): Dropout(p=0.0, inplace=False)
  (tau): Softplus(beta=1.0, threshold=20.0)
  (V_d_transform): Identity()
  (graph_transform): Identity()
)