Skip to content

Instantly share code, notes, and snippets.

@Equim-chan
Last active February 23, 2024 09:56
Show Gist options
  • Star 24 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save Equim-chan/cf3f01735d5d98f1e7be02e94b288c56 to your computer and use it in GitHub Desktop.
Save Equim-chan/cf3f01735d5d98f1e7be02e94b288c56 to your computer and use it in GitHub Desktop.
Should I release the trained model of my mahjong AI?
简体中文

更新

2022-08-19

结论:鉴于在线版本 https://mjai.ekyu.moe 已经基本实现了我所期望自己的 AI 去提供的功能,目前没有公开模型参数的计划。

2022-04-13

已经开源在了 https://github.com/Equim-chan/Mortal

我是否要发布训练好的麻将 AI 模型?

麻将社群,我需要你们的帮助。

背景

我是 akochan-reviewer 的作者,这是一个基于 critter 的 akochan 制作的麻将学习工具

我制作麻将牌谱检讨工具的初衷是希望它能成为一个 7x24 小时在线的教练来教导人类玩家,尤其是那些没法接触到高质量教学资源或者教练的人。这样的工具已经在国际象棋和围棋这样的游戏里存在了,而麻将也应当有一个。当我看到有人说他们从 akochan-reviewer 学到了什么的时候,我是挺欣慰的,即便 akochan 时常做出反常的行为。

akochan 举止怪异是一个我也无法修复的已知问题。出于对 akochan 存在的种种限制、不足、以及糟糕的性能的不满,我决定自己做一个全新的 AI。

费劲将近一年的心血之后,我从零开始完成了一个强度足够的麻将 AI 的 1.0 版本。它很强,根据它和 akochan 对战的统计数据可以得出它的水平已经超过 akochan,也包括了其他我所能拿到的 baseline,它可能和初版的 Suphx 强度相当。

这个 AI 兼容 akochan 的协议,也就是 mjai,所以我的 akochan-reviewer 能很轻易地支持以它为后端。

在接下来的几个月,我准备要将源代码公开至 GitHub,但我在犹豫是否要公开训练好的模型。

如果你感兴趣的话,这里有几张新 AI 的 1.0 版本(旧版本,现已被新版本击败)之间自我对战的牌谱:

理由

公开模型的理由

1. 有始有终

如果只有代码没有预先训练好的模型,对于一个深度学习项目来说总显得有点不太完整。如果模型也发布了的话,其他的研究员就无需自己训练模型了,节省了大量的时间和金钱。

2. 在本地运行的能力

正如我所做的 akochan-reviewer 一样,用户可以轻易地下载到预先构建好的版本,然后在本地跑。目前,我正在维护的 akochan-reviewer 的在线版本只有有限的功能,也无法检讨雀魂牌谱,而本地版本却有所有的功能,并且能读取下载好的雀魂牌谱,甚至还能输入任意自定义的牌谱。

3. 方便衍生作品

也许其他人比我有更好的方式去做麻将牌谱检讨机。发布模型的话,他们就可以轻易地发布或部署他们自己的检讨机或者其他正规的衍生作品,只要把模型嵌入。

4. 就算不发布,开挂的人自己也会训练模型

然后他们还能以此为理由卖得更贵。

也许在我把源代码公开的那一刻,我就已经输给了做挂的人。如果他们不惜一切代价都要搞的话,说不定我自己免费公开模型让他们难以获利还更好,尽管这理由听着挺蠢。

不公开模型的理由

开挂的人一定会利用它

这是我最大的担忧。

首先有个坏消息,这样的不当利用 已经 发生在了 akochan 上。一些脚本小子已经售卖基于 akochan 的雀魂 bot 有些时间了。尽管他们在描述里没提到 akochan 的名字,但有很多线索指向他们在用 akochan 或者 akochan 的改版。

我看到的时候心情很复杂。我对做挂卖挂的行为感到气愤,也为他们在卖偷来的东西感到失望。但突然间又感到一丝庆幸,还好他们没打上 akochan 的标签,不然吃瓜群众可能会误解是 akochan 开发者的授意。最后我只能感到悲哀,因为即便他们没有售卖、散布、制作外挂,终有一天也会冒出来另一群人在做类似的事,只要 AI 能够从公共途径获取到。

我并不反对制作天凤或者雀魂的个室 bot,相反,我觉得他们非常实用,无论是想玩但找不齐人,或者只是单纯好奇想和 AI 在自己熟悉的平台上比试比试。

可惜,人的恶是无法被阻止的。能被不当利用的东西一定会被不当利用。如果我发布了训练好的模型,我实在找不到什么方法能去避免它被利用去打段位战。更糟糕的是,我可能还会被受害者指责“在帮人做外挂”。很难想象会有多糟糕。

其他类似的项目是如何处理的?

国际象棋的 stockfish

stockfish 是个很有名的开源国际象棋 AI。如果你玩国际象棋,辛苦把你所知道的分享一下。

  • stockfish 有把在线象棋平台毁了吗?如果是的话是多大程度的影响?
  • 在线象棋平台如何避免人们使用 stockfish 来作弊?
  • 在线象棋的玩家对此怎么看?

NLP 的 GPT-3

这个不是游戏 AI,但它在现在这个上下文里算是个不错的例子。

OpenAI 的 GPT-3 的预训练模型,从来没有被公开发布过。根据我所听到的故事,是因为模型实在太强,太容易被滥用。当然也可能是因为商业价值太高。

最近,OpenAI 宣布了一系列的由 GPT-3 模型支持的在线服务。我觉得我也可以走类似的路子,提供些在线服务,比如牌谱检讨和去个室玩。这也是 NAGA 正在做的事情。

可能的解决方案

1. 不发布模型

最直接,但这很大程度上限制了这个项目能对麻将社群做的贡献。

2. 不发布模型,但是提供在线服务

听起来不错,但是开发和维护在线服务需要时间和钱,而且如果我哪天丧失了兴趣就没了。我可能需要做一些需要捐赠才能解锁的会员功能来支撑服务运行下去。但是做成收费的话又会丧失其之于 NAGA 的优势。

3. 发布模型,但是要付费

盗版的运作方式同样能放在这。同时,这也只让做外挂的人更难搞,并不是完全阻止。

… 更多?

请在下方评论。


如果你有任何想法,请在下方留言。谢谢。

Equim.

English

Updates

2022-08-19

Conclusion: considering the online version https://mjai.ekyu.moe has already provided what I expected my AI to provide, currently I have no plan to release the trained model.

2022-04-13

The source code is now avilable on GitHub at https://github.com/Equim-chan/Mortal.

Should I release the trained model of my mahjong AI?

Dear mahojng community, I need your help.

Background

I am the author of akochan-reviewer, a mahjong learning tool powered by critter's akochan.

My original intention for building mahjong reviewer was that it could serve as a 7x24 available coach that guides human players, especially for those without access to high-quality tutorial or coach resource. Such tools already exist in other games like chess and go, and mahjong should also have one. I am really happy to see people telling me they learn something from akochan-reviewer, even though akochan acts weird from time to time.

Akochan acts weird, this is one of the known issues, and I can't fix it as well. Out of dissatisfaction with the limitations, weaknesses and poor performance of akochan, I decided to make a brand new one myself.

After almost a year of hard work, I managed to build the 1.0 version of a powerful mahjong AI from scratch. It is very strong. Statistics from test plays conclude it is stronger than akochan and any other available baseline I have, and it's probably as strong as the early version of Suphx.

The AI is compatible with akochan's protocol, namely mjai, therefore it can serve as a backend for my akochan-reviewer very easily.

In the next several months, I am going to release the source code on GitHub, but I am hesitant to release the trained model.

If you are interested, here are some games the new AI (version 1.0, weaker than the current version) played with each others:

Reasons

To release the model

1. Make the research complete

It feels a bit incomplete for a deep learning project to only release the code without a trained model. If the model is released, other researchers won't have to reproduce the model on their own, which save a huge amount of time and money.

2. Ability of running locally

Just like akochan-reviewer I made, users can easily download a prebuilt all-in-one package and run it locally. At the moment, the online reviewer service I maintain has limited features and no access to mjsoul logs, whereas the local version has all the features and can read downloaded mjsoul logs and even any hand-crafted log.

3. Easier for derived works

Maybe somebody else have a better idea of making a mahjong reviewer than me. By releasing the trained model, they can easily release or deploy their own reviewer or other legit derived works bundled with the AI.

4. Cheaters are going to train their own model anyways

And they will even sell it higher for such reason.

I may have already lost to cheaters by the time I release the code to the public. If they're going to do it at any cost, it might be even better for me to release the model for free to make them harder to profit, although the idea sounds a bit dumb.

Not to release the model

Cheaters will definitely exploit it

This is the biggest concern.

First of all I have some bad news, such exploit is already happening on akochan. Some script kiddies in the wild have been selling akochan bot for mjsoul for some time. Although they don't mention of the name "akochan" in the description, there are some indications that they are using akochan or a modified version of it.

I had very mixed feelings when I saw it. I was angry for the conduct of making and selling cheats itself, and also for them selling things they stole from others. But then I happened to feel a bit thankful for them not having tagged it with the name "akochan", which will be worse as people may think it is affiliated with akochan devs. And in the end, I was just sad. Even if those guys had not sold, distributed, or even developed the cheat, as long as the AI is publicly available, another group of people will eventually get their hands on it.

I am not against making tenhou or mjsoul bots that play in private lobbies, instead, I think they are very handy and useful when you want to play but can't find enough friends, or you are just curious and simply want to challenge the AI on the platform you are familiar with.

Unfortunately, there is no way to stop people from being evil. Things that can be exploited will be exploited. If I release the trained model, I don't see a way to prevent it from being exploited to play in ranked lobbies. Even worse, I may become the one to be blamed by some victims for "assisting cheaters". I honestly can't imagine how horrible it will be.

How do other similar projects handle it?

stockfish for chess

Stockfish is a well-known open source chess AI. If you are a chess player, please share information you know about it.

  • Has and how has stockfish destroyed online chess?
  • How do online chess platforms keep people from cheating with stockfish?
  • How do online chess players think about it?

GPT-3 for NLP

This is not a game AI, but it serves as a good example in this context.

The trained model of GPT-3, by OpenAI, were never released to the public, according to the story I heard, because the model is so powerful that it can't be kept from being exploited. Of course it is also possible that it is because the commercial value of it is too high.

Recently, OpenAI announced a suite of online services backed by GPT-3. I believe I can go the similar way by not releasing the weights and instead offering online reviewer and private lobby bot services. Offering online services is also what NAGA is doing.

Possible solutions

1. Do not release the model at all

The straightforward way, but it greatly limits the project's ability to benefit the mahjong community.

2. Do not release the model, but offer an online service for review and bot for private lobby

Sounds good, but it takes time and money for me to develop and maintain the service, and it probably won't last forever if someday I lose my interest. I may need to make some features premium that require donations to unlock to keep the service alive. But if it becomes pay-to-use, it will lose its advantage over NAGA.

3. Release the model, but behind a paywall

We all know how piracy works and the same can apply to this. It only makes it harder for cheaters but does not completely make it impossible.

... more?

Please comment down bellow.


If you have any idea, please comment down bellow. Thank you.

Equim.

@Zeks
Copy link

Zeks commented Sep 5, 2022

Now that the site has been purged by tenhou everyone regrets the decision to never release the model.

@gavingrey
Copy link

I find it disappointing that there is no trained model and the documentation is scrubbed of information. In the current state this project appears to be an advertisement for an analysis website. I would be eager to contribute but not without documentation

@jacy
Copy link

jacy commented Jul 31, 2023

how can I start training the AI to play like a normal human(not need to be smarter than human, but not a dumbheaded AI either)

@jacy
Copy link

jacy commented Aug 3, 2023

hope moral can open source completely like doudizhu

https://github.com/kwai/DouZero

@nideii
Copy link

nideii commented Sep 19, 2023

TL;DR
Very simply, grow the community so large that those for profit companies have to build anti-cheat in order to make more money.

First, let's talk about the technical details, and luckily, there is a chess boom happening.
One of, or the, largest chess website, chess.com, use deep learning method to find cheaters. I am not 100% sure what model they used, but it might be a stockfish variant. And for human to identify a cheater, beside cheaters' move are very similar to stockfish, the time between each move also be very consistent.

HOW TO IDENTIFY A CHESS CHEATER,

  1. Cheaters might make bad move at the beginning, thinking they might get away with this.
  2. The time the spend for each move is very consistent.
  3. They might not always choose the top stockfish move.
  4. When remaining time is a big factor, like during the end game, they will lose on time or make bad move.
  5. They usually make new accounts

Here is my idea about cheating and tooling,

First, if you don't release the model, then yes, few people will use it to cheat, and much more people cannot use this to learn, as long as there are more honest player than cheaters.
Second, stop releasing the model, then cheaters will find other ways to cheat, they will not stop cheating because this tool is not available.
Third, there will eventually be a new and better deep learning models tomorrow to replace STOA models today. Make everything more assessable to everyone, this process will happen much faster.

It is good to think about cheating when making tools. However, it is not single individual's responsibility to prevent cheating or stop cheating, everyone does.

Just think about this, if you solve world hunger, but afraid some dictators will start a war for it, so you choose not to release it, then who benefit the most? You? the dictators? or no one?

@ashleycolton
Copy link

it is already out there with a paywall. and there's currently no alternative open source mahjong ai..
as far as i know, these projects are already filled with AI:
chess
GO
poker
gomoku
mahjong

i'm hoping to see an open source one tbh

@BarclayII
Copy link

BarclayII commented Feb 23, 2024

One necessary component I feel is missing for mahjong in particular is explainability.

We know that stockfish is open source and is probably the best AI in chess, but it doesn't offer comprehensive explanations to its policy. However, there are some platforms that offers explanations in natural language, indicating which moves are brilliant, good, or poor, as well as the reasons behind such judgement (e.g. "you lose a knight if you play this move"). Chess.com is a good example - I used it for a while, and it even offers a "what to improve" (e.g. endgames, forks, etc.) section after a review. The paywall is super expensive.

I didn't see a route to explainability in mahjong RL agents as clear as those for chess though, primarily because of two reasons: (1) the inherent randomness, and (2) the partially observable nature. Even with standard "nanikiri" problems where you are only given your hand without any other information, I still saw some of the professional players disagree on what is the best policy by saying "this may happen", "that may happen", etc., and often times agree to disagree. So if one indeed builds such an explainability service, it will become a stable profit source and many more people will benefit from it, even without releasing the model weights or even the architecture.

I have some thoughts on how an explainability model for mahjong RL agents may look like, but I didn't get the time to formulate it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment