* !markov help will display a pretty crude help message. It's funny to
me, I dunno.
* help_timeout config option limits the number of times that `!markov
help` is allowed to run. This is because the help message is a lot of
lines, and it could cause problems if `!markov help` is spammed or
abused. It can be set to 0 or less to disable.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
Last patch was creating a set of the list of all matches, instead of a
set of all matches. This fixes that, and also makes the regex a raw
string.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
If there was an unmatched word 'foo', and someone wrote 'foo. bar.', wordbot would not recognize the match because it was only splitting the string on spaces, not punctuation.
This treats as a word any contiguous sequence of letters, numbers (just in case) and hyphens (just in case).
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
The allchain has been a source of headaches because it takes up a lot of
memory and slows everything down. However, with the new database
model, we can generate markov sentences using all of the rows since they
are a flat collection. This helps reduce disk space and increases the
import speed significantly.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
Markov now uses a sqlite3 database instead of flat JSON files. This
should significantly speed up saving time, plus reduce the amount of RAM
that it uses. Saving and loading large JSON files was very slow and
caused issues with other plugins, especially when messages were
received. Additionally, in order to save RAM, a cache was used and
periodically flushed when not used, adding some complications to the
implementation. This has all been removed since things get committed on
the fly with the database implementation. The main trade-off we have to
make is the disk space used by the database. This is OK though, because
disk space is cheap while RAM is not.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This is a lot simpler from a concurrency perspective. Training values
can get committed to the database immediately, rather than in
long-running flat file batches.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
If an HTML title was parsed with whitespace, it would not strip that
surrounding whitespace. This fixes that.
Also, there are some new debug log messages in linkbot. Hooray!
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
Just like the actual chain data structure, this value is now loaded
lazily, since it's stored in the filesystem.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This allows markov to save (hopefully) in parallel using a
ProcessPoolExecutor. Since objects are sent over-the-wire and copied,
pruning in parallel is not an issue.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This moves the self.__touch() call around in markov's Chain class such
that it will only access truly available data.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
* Chain.__touch() is a new function that updates the last time a markov
chain was accessed
* Fix a bug that would not reliably update the last access time of the
chain during Chain.add()
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
Markov chains used to prune the chains themselves from memory, but now
that behavior is specifically delegated up the chain to the Bot
structure instead.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
* Linkbot parser also looks for <meta> tags and uses an actual HTML
parser.
* Inner title HTML is decoded before being displayed.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
When a user's name is used in the !wordbot leaderboard command, we make
every effort to not ping them by interleaving zero-width space
characters in the nickname.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This matches the old database format that was written a while back.
There's an "end_now" command that's been left in there for debugging
purposes, that'll be gone soon enough.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
* .items() call required when loading a markov chain into memory
* `who.nick` instead of `who~ for get_chain call
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This will just send a message to the user who just updated their markov
reply chance with the final value it was set to, so there are no
surprises.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
Whenever someone says something, there's a chance that markov will
interject his opinion. Users can also set the chance between 0.0 and the
default value (in the config) if they want to see markov replies less
often.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
If something is changed in a markov chain it gets flagged as dirty,
which is used to determine whether the chain should be saved.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
If you don't use/access your chain every N seconds (300 by default), it
will unload your chain from memory and save it to disk.
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This is beneficial for all plugins so the bot doesn't accidentally
spam things because the plugin writer didn't check their inputs
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
