How to initialize network to output zeros?

ElLoboLoco · April 21, 2021, 8:08pm

Hi,

I’m currently trying to implement the algorithm found in this paper : https://arxiv.org/pdf/1811.00164.pdf.

However, at some point, the algorithm need to initialize the neural network to return 0 for all inputs (line 1).

So my question is simply how do I do that ? At first I thought to initialize the weights of the output layer to zero but it will prevent it from learning. I’m sure there is a simple method but I can’t find it.

googlebot · April 22, 2021, 3:18am

It won’t, this only applies to hidden layers (that generate “artifical” features). For output layer, loss (eq.: distance to target) provides a non-zero gradient.

ScottD · December 13, 2021, 12:25am

Hi, I was wondering if you had any luck figuring this out? I came across this topic because I was actually trying to implement that same algorithm, and I had the same question.

I tried your suggestion regarding the output layer weights, and didn’t get any improvement in performance. My advantage networks end up with a loss of around 30k mean squared error and I converge around 500mbb exploitability. I can’t figure out where I’m going wrong in replicating the neural net of the paper, but this is one line that I don’t understand. I’ve never seen such a technique discussed anywhere.

ElLoboLoco · December 15, 2021, 12:27pm

Hi, initializing the weights of the output layer to zero seemed to work in my case. You can still look on public implementation of deep cfr such as :

github.com

deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py

# Copyright 2019 DeepMind Technologies Ltd. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Implements Deep CFR Algorithm.

See https://arxiv.org/abs/1811.00164.

The algorithm defines an `advantage` and `strategy` networks that compute
advantages used to do regret matching across information sets and to approximate

This file has been truncated. show original

or

Hope that helps