liangxuZhang e768616afa Fix load balancing loss func for mixtral (#28256) ...

* Correct the implementation of auxiliary loss of mixtrtal

* correct the implementation of auxiliary loss of mixtrtal

* Implement a simpler calculation method

---------

Co-authored-by: zhangliangxu3 <zhangliangxu3@jd.com>

2024-01-11 16:16:12 +01:00

..

__init__.py

[Add Mixtral] Adds support for the Mixtral MoE (#27942)

2023-12-11 12:50:27 +01:00

test_modeling_mixtral.py

Fix load balancing loss func for mixtral (#28256)

2024-01-11 16:16:12 +01:00