The derivative of x->x (identity) should be 1 regardless of your definition of derivative.
Subgradients are only applicable when summing convex functions. Here relu(x) - relu(-x) is a sum of a convex function and a concave function.
Subgradients are only applicable when summing convex functions. Here relu(x) - relu(-x) is a sum of a convex function and a concave function.
No comments yet.