Make a cyrillic chat...

Afijak

So a proposition is like that in the title - how about to make a switch language option ("Shift+Alt" like in Windows) to make possible cyrillic typing in the ingame chat (for example in russian)?? I dont use microphone and often use russian servers (because im a native russianspeaker) and lack of cyrillic symbols is a trouble for me...

rkfg

That's not about cyrillic only, I guess. Unicode support would be great for everyone and it would require no specific keys to switch layouts.

Omega_K2

rkfg wrote: »

That's not about cyrillic only, I guess. Unicode support would be great for everyone and it would require no specific keys to switch layouts.

Good idea.

LSR

Went on forums to suggest the same, but found this article so BUMP.

Maxx11_v2.0

Im sure this is something that could easily be, or already is, handled by a mod.

rkfg

So, more than a year has passed. I think there's no reason to start a new thread about the same old issue so here's my findings. I grepped the lua source and found the root of all input processing, it's called InputHandler.lua. It has two main methods that process the keyboard input: OnSendKeyEvent and OnSendCharacterEvent. The first handles scancodes and thus does not depend on the kb layout, it gets the same numbers no matter what layout I use. The second receives processed characters. and it actually is not called at all when I enter any cyrillics. An obvious idea is to simulate the call of OnSendCharacterEvent from OnSendKeyEvent supplying corresponding cyrillic key mapped by a scancode. However, there are two problems:
1) A basic approach would only allow entering lowercase (or only uppercase) letters as scancodes are on lower level than Shift/CapsLock modifiers. It would require special handling of Shift key instead of offloading this task to the OS. But that's quite negligible.
2) The main problem is that OnSendCharacterEvent receives rather a cryptic data instead of a character. It looks like "userdata 0x8192abcd" (i.e. a hexadecimal number). Then it gets to GUIChat:SendCharacterEvent (at GUIChat.lua) where it's being converted to an actual one-character string and added to the chat GUI control. The conversion is also kinda hacky (actually, ugly), it's at Utility.lua and called ConvertWideStringToString(). It creates a GUI item (probably, some base class for all GUI elements), uses SetWideText() on it and then gets the contents as text via GetText(). Why use a GUI element when you can export an engine function directly with less overhead? Anyway, we can't craft such a structure from lua because it's a special data type that can't be accessed from scripts. It's an ideal black box. Period.

Okay, there IS a way to enter cyrillics, I think. For example, there can be a special internal layout flag that, if set, translates entered latin chars into corresponding cyrillics. For example, a user type "ghbdtn", the chat decodes it into "привет" using a simple char replacement according to the layout. Though these chars would be invisible anyway for Linux players (that's another issue). But it's worth a try.

Overall, the problem is that the engine doesn't call OnSendCharacterEvent() if the layout is not English. This particular issue seems to be impossible to fix from just lua.

GhoulofGSG9

rkfg wrote: »

So, more than a year has passed. I think there's no reason to start a new thread about the same old issue so here's my findings. I grepped the lua source and found the root of all input processing, it's called InputHandler.lua. It has two main methods that process the keyboard input: OnSendKeyEvent and OnSendCharacterEvent. The first handles scancodes and thus does not depend on the kb layout, it gets the same numbers no matter what layout I use. The second receives processed characters. and it actually is not called at all when I enter any cyrillics. An obvious idea is to simulate the call of OnSendCharacterEvent from OnSendKeyEvent supplying corresponding cyrillic key mapped by a scancode. However, there are two problems:
1) A basic approach would only allow entering lowercase (or only uppercase) letters as scancodes are on lower level than Shift/CapsLock modifiers. It would require special handling of Shift key instead of offloading this task to the OS. But that's quite negligible.
2) The main problem is that OnSendCharacterEvent receives rather a cryptic data instead of a character. It looks like "userdata 0x8192abcd" (i.e. a hexadecimal number). Then it gets to GUIChat:SendCharacterEvent (at GUIChat.lua) where it's being converted to an actual one-character string and added to the chat GUI control. The conversion is also kinda hacky (actually, ugly), it's at Utility.lua and called ConvertWideStringToString(). It creates a GUI item (probably, some base class for all GUI elements), uses SetWideText() on it and then gets the contents as text via GetText(). Why use a GUI element when you can export an engine function directly with less overhead? Anyway, we can't craft such a structure from lua because it's a special data type that can't be accessed from scripts. It's an ideal black box. Period.

Okay, there IS a way to enter cyrillics, I think. For example, there can be a special internal layout flag that, if set, translates entered latin chars into corresponding cyrillics. For example, a user type "ghbdtn", the chat decodes it into "привет" using a simple char replacement according to the layout. Though these chars would be invisible anyway for Linux players (that's another issue). But it's worth a try.

Overall, the problem is that the engine doesn't call OnSendCharacterEvent() if the layout is not English. This particular issue seems to be impossible to fix from just lua.

Just wait for the next patch currently the engine uses as std locale only "en_US" which is just the basic ASCII charset, we changed that for 273 to "en_US.utf"8 and so far all playtesters reported that their issues with non ascii keys are gone (same for the linux issue of displaying those chars).

The only issue left would be that the lua key map is still only ASCII and therefor non ASCII keys have weird name in the the option menu (like Key #367) but that shouldn't be a large issue.

rkfg

GhoulofGSG9 wrote: »

Just wait for the next patch currently the engine uses as std locale only "en_US" which is just the basic ASCII charset, we changed that for 273 to "en_US.utf"8 and so far all playtesters reported that their issues with non ascii keys are gone (same for the linux issue of displaying those chars).

That's great news! So do I understand it right, we'll be able to both see non-ASCII chars and type them? Because Rust, for instance, displays cyrillics just fine but typing them shows arabic script (literally). Hope this will be resolved for both ways.

GhoulofGSG9 wrote: »

The only issue left would be that the lua key map is still only ASCII and therefor non ASCII keys have weird name in the the option menu (like Key #367) but that shouldn't be a large issue.

Indeed.

P.S. I'll hack my little mod anyway, it's already working actually.

rkfg

Guess what? I've fixed this thanks to the locale tip. So for everyone who wants cyrillics and other non-ASCII characters right now, here's how to do this. First, read this post, it describes how to use preloads. Now compile this code:

// preload.c
// Alistair Buxton <a.j.buxton@gmail.com>

#define _GNU_SOURCE

#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>

void __attribute__ ((constructor)) load(void);


// Called when the library is loaded and before dlopen() returns
void load(void)
{
    fprintf(stderr, "preload hack loaded...\n");
}

typedef char* (* setlocale_func)(int category, const char *locale);

char *setlocale(int category, const char *locale) {

    static setlocale_func slf = NULL;

    if(!slf) {
        slf = dlsym(RTLD_NEXT, "setlocale");
    }
    return slf(category, "en_US.UTF-8");
}

It's almost the same but I replaced the glBindBuffer function with setlocale that's being intercept and then called with the right locale parameter. You can use the same makefile to build it, then put the libnvidiathing.so file to the NS2 directory and set the launch parameters in Steam to LD_PRELOAD=$LD_PRELOAD:libnvidiathing.so %command%
...or whatever the resulting .so library is called.

Magically, it enables unicode and all non-ASCII symbols become visible in chat as well as the game now allows typing them. This will also enable localizations for all languages:

Drem_Luck

TO rkfg
Your guide is designed for experienced users, and yet it does not say that the library libnvidiathing.so to compile on x32 system
Инструкция для x32-битных Linux систем.
Чтобы корректно руссифицировать Natural Selection 2, нужно.
1. Перейти в раздел вашей игры (например у меня /.steam/steam/steamapps/common/Natural Selection 2/)
2. Создать файл preload.c, в нем вставить текст

// preload.c
// Alistair Buxton <a.j.buxton@gmail.com>

#define _GNU_SOURCE

#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>

void __attribute__ ((constructor)) load(void);


// Called when the library is loaded and before dlopen() returns
void load(void)
{
fprintf(stderr, "preload hack loaded...\n");
}

typedef char* (* setlocale_func)(int category, const char *locale);

char *setlocale(int category, const char *locale) {

static setlocale_func slf = NULL;

if(!slf) {
slf = dlsym(RTLD_NEXT, "setlocale");
}
return slf(category, "en_US.UTF-8");
}

3. Создать файл Makefile, в нем вставить текст

all: libnvidiathing.so

%.o: %.c
gcc -m32 -fPIC -c -Wall $*.c -o $*.o

libnvidiathing.so: preload.o
gcc -m32 -shared preload.o -ldl -o libnvidiathing.so

clean:
rm -f *.o *.so

4. В текущей директории открыть терминал и ввести следующие команды:
gcc -fPIC -c -o preload.o preload.c
gcc -shared -o preload.so preload.o -ldl
make Makefile all
5. В Steam в Библиотеке выбрать Natural Selection 2 -> Свойства -> Установить параметры запуска и ввести
LD_PRELOAD=$LD_PRELOAD:libnvidiathing.so %command%
В игре в Настройках выбрать Русский

rkfg

I linked the Makefile you can use, it already has -m32 flag enabled so it should build 32-bit version. You're also welcome to ask things you don't know or understand, Linux world is huge and you're free to learn things and help others. Also, these were 3 hours well spent, you've learned something new the hard way!

But anyway this hint (or "guide") gets outdated very fast, build 273 was announced to be released today and it should include the locale fix.

Drem_Luck

Maybe I learned something new. But how much was spent nerves! And said bad words against developers! Still, who can and need my instruction.

GhoulofGSG9

Guys glad to see it works but just wait a few hours. 273 is close to hit the door and will be released today

rkfg

If this really gets on your nerves, there's no need to curse devs or the system. Just ask. Linux is overall about learning and hacking, it's not "free Windows". You can happily live without such knowledge as compiling stuff but one day it may become pretty handy. The problem is, compiling is really a basic thing in Linux so I haven't thought it should be explained in details. You haven't explained how to "В текущей директории открыть терминал" (open the terminal in current directory), not every DE allows to do this easily so it requires cd'ing to some directory and hence knowing the shell basics. I haven't mentioned where to look for the NS2 root directory, too, for me it's obvious. I don't need to say all of this explicitly, nor need you. But someone else does. My point was not to teach people how to use make and gcc (it's googleable) but to provide a working hack. Hack implies the ability to do hacking which is something you definitely may expect from a Linux user rather than from a Windows user.

Also, you don't need to execute this:

gcc -fPIC -c -o preload.o preload.c
gcc -shared -o preload.so preload.o -ldl

It's done via Makefile, and in this form (without -m32) it will produce a 64-bit library named "preload.so" which you don't use afterwards.

"make Makefile all" can be replaced with just "make".

That said, I understand your frustration and I don't mock you. I'm glad you were able to figure it all out.

А если вопросы всё же остались, спрашивай в нашей группе. Линуксоидов там исчезающе мало, так что игнором не разбрасываемся, по мере сил поможем!

Drem_Luck

TO rkfg
huh

Твое подробное объяснение о том, как пользоваться Makefile пригодилось бы пораньше XD. Я недавно перешел на Linux, поэтому некоторые вещи не очень понятны. Да и, думаю, не все пользователи форума хорошие программисты. Есть люди, которые просто хотят поиграть и не напрягаться (вот только на Linux играть и не напрягаться - две разные вещи XD. Чего стоит пляска с бубном над видеокартой и драйверами под нее XD