PokerPirate
25d ago
I've never heard of mLSTMs before, and googling led me to https://arxiv.org/abs/1609.07959. I don't see anything about parallel training in the abstract here, and it was also released 1 year before AIAYN.
Can you link me to what you're actually talking about?
(And as an aside, I don't see a meaningful way to parallel train LSTMs without actually converting it into a transformer architecture "in disguise". The whole point of LSTMs is that the rightmost token depends on the leftmost token, and so any training that allows parallelism fundamentally breaks this property (unless P=NC---which is not a typo for NP).
Thank you!
More options
Context Copy link