No Unicode Character will print with NeoVars #550

Open
opened 10 months ago by Polyhistorian · 10 comments

I've managed to narrow down the issue enough that any character which goes into a SendUnicodeChar methdod refuses to appear anywhere in any form. I can confirm the character code getting to the functions in question, but they are too complicated for me try and debug any further. Most critically the character "^" won't print itself, which is critical in at least some programming languages. This seems to affect any such character, whether assigned to a key, created with dead keys, or created through compose.

I've managed to narrow down the issue enough that any character which goes into a SendUnicodeChar methdod refuses to appear anywhere in any form. I can confirm the character code getting to the functions in question, but they are too complicated for me try and debug any further. Most critically the character "^" won't print itself, which is critical in at least some programming languages. This seems to affect any such character, whether assigned to a key, created with dead keys, or created through compose.
Owner

More context please.

More context please.

Neo-Vars: Newest from repo, built with the included scripts into an exe. Though the bug also persists with the loose scripts.

Windows Pro: Version 10.0.1904

AutoHotkey: v1.1.33.01 (Just updated to v1.1.33.02, the bug still replicates itself)

Anything else wanted?

Neo-Vars: Newest from repo, built with the included scripts into an exe. Though the bug also persists with the loose scripts. Windows Pro: Version 10.0.1904 AutoHotkey: v1.1.33.01 (Just updated to v1.1.33.02, the bug still replicates itself) Anything else wanted?
qwertfisch added the
Treiber/Windows/AHK
label 10 months ago

Update on the "^" character situation. I haven't found anything at all pointing to an overall fix. But by changing the caret chacter on the line 26 of keydefinitions.ahk into the caret that is needed (the characters won't copy so that the difference would be visible). And then uncommenting line 402 or line 602 depending on your layout. (If someone is unsure, you can uncomment both to be sure). That produces valid output in every situation that I've tested. Not sure why they are commented, there's probably a good reason, but as a work around it functions a 100% for me.

Update on the "^" character situation. I haven't found anything at all pointing to an overall fix. But by changing the caret chacter on the line 26 of [keydefinitions.ahk](https://git.neo-layout.org/neo/neo-layout/src/branch/master/windows/neo-vars/src/keydefinitions.ahk#L26) into the caret that is needed (the characters won't copy so that the difference would be visible). And then uncommenting [line 402](https://git.neo-layout.org/neo/neo-layout/src/branch/master/windows/neo-vars/src/keydefinitions.ahk#L402) or [line 602](https://git.neo-layout.org/neo/neo-layout/src/branch/master/windows/neo-vars/src/keydefinitions.ahk#L610) depending on your layout. (If someone is unsure, you can uncomment both to be sure). That produces valid output in every situation that I've tested. Not sure why they are commented, there's probably a good reason, but as a work around it functions a 100% for me.
Owner

From the initial bug report, you describe what you think caused the issue, but there is no description of what the problem really is. Can you add some information about the problem itself, possible steps to reproduce, what you would expect, and what instead happens?

The NeoVars driver works for me and for dozens of other people, without problems on displaying any character. The caret symbol (you probably are referring to U+005E „Circumflex accent“) will be displayed on Mod3+W (for Neo, or „T“ on qwertz keyboards).

Last but not least, if German is your native language you are also free to continue here in German.

From the initial bug report, you describe what you think caused the issue, but there is no description of what the problem really **is**. Can you add some information about the problem itself, possible steps to reproduce, what you would expect, and what instead happens? The NeoVars driver works for me and for dozens of other people, without problems on displaying any character. The caret symbol (you probably are referring to U+005E „Circumflex accent“) will be displayed on Mod3+W (for Neo, or „T“ on qwertz keyboards). Last but not least, if German is your native language you are also free to continue here in German.

The problem is for me that any Unicode function character just doesn't appear anywhere. I 've tried firefox, notepad, visual studio code, notepad++, windows search, none of them get any input from the unicode functions. Not a broken character, or even a blank character, seemingly nothing comes out, the possible opened file doesn't even get marked as changed.

The problem reproduces itself with any dead character combined character, compose, and possible assigned to key Unicode function characters. As it did with the caret for me until my "fix", if you can call it that. So: Text field — > Try to enter any Unicode function character — > No output.

Yeah, I was referring to that character, the keybind just never printed it for me. I got it to work by uncommenting line 417 in the keydefinitions.ahk file. But it resulted in a broken character for me, not sure why, so I went with the "fix" I previously described.

Unfortunately, I don't know a word of German, I'm from Finland personally. So there's technically a possibility it somehow relates to display language settings, but I can't see why it would be so. The keyboard is by layout QWERTY but extended to the nordic layout, which does seem to match QWERTZ other than the placement of characters, which I've corrected (at least mostly) with other changes to the keydefinitions.ahk file. And even if the key in question wouldn't happen to work it doesn't explain compose or other dead keys not working as well, including the ones I haven't touched.

The problem is for me that any Unicode function character just doesn't appear anywhere. I 've tried firefox, notepad, visual studio code, notepad++, windows search, none of them get any input from the unicode functions. Not a broken character, or even a blank character, seemingly nothing comes out, the possible opened file doesn't even get marked as changed. The problem reproduces itself with any dead character combined character, compose, and possible assigned to key Unicode function characters. As it did with the caret for me until my "fix", if you can call it that. So: Text field — > Try to enter any Unicode function character — > No output. Yeah, I was referring to that character, the keybind just never printed it for me. I got it to work by uncommenting [line 417](https://git.neo-layout.org/neo/neo-layout/src/branch/master/windows/neo-vars/src/keydefinitions.ahk#L417) in the keydefinitions.ahk file. But it resulted in a broken character for me, not sure why, so I went with the "fix" I previously described. Unfortunately, I don't know a word of German, I'm from Finland personally. So there's technically a possibility it somehow relates to display language settings, but I can't see why it would be so. The keyboard is by layout QWERTY but extended to the [nordic layout](https://en.wikipedia.org/wiki/QWERTY#/media/File:KB_Finnish_Multilingual.svg), which does seem to match QWERTZ other than the placement of characters, which I've corrected (at least mostly) with other changes to the keydefinitions.ahk file. And even if the key in question wouldn't happen to work it doesn't explain compose or other dead keys not working as well, including the ones I haven't touched.
hrnz changed title from [Neo-Vars] [Win 10] No Unicode Character will print. to No Unicode Character will print with NeoVars 3 months ago

The issue described by @Polyhistorian fits a scenario where neovars runs on a 64-bit version of AutoHotkey (AutoHotkeyU64.exe).

Current neovars is incompatible with 64-bit and will fail to send out any unicode characters, while ASCII still works. This might explain why the fix described by @Polyhistorian works: It replaces certain unicode characters with ASCII symbols.

(Additionally, keyboard LEDs will not work in 64 bit)

@Polyhistorian, if you are still interested:

  • As a test and possible solution, run the neo20-all.ahk explicitly with the AutoHotkeyU32.exe which can be found in the Autohotkey program directory.
  • In case you run a compiled neo20.exe, you should try to recompile the sources explicitly with the 32-bit binaries (Unicode 32-bit.bin).
  • Also make sure sure that you do not run the ASCII version of Autohotkey (AutoHotkeyA32.exe), which is not supported either.
  • Better yet, just fetch the latest version of the repo and recompile (see @qwertfisch comment below)
The issue described by @Polyhistorian fits a scenario where neovars runs on a 64-bit version of AutoHotkey (AutoHotkeyU64.exe). Current neovars is incompatible with 64-bit and will fail to send out any unicode characters, while ASCII still works. This might explain why the fix described by @Polyhistorian works: It replaces certain unicode characters with ASCII symbols. (Additionally, keyboard LEDs will not work in 64 bit) @Polyhistorian, if you are still interested: * As a test and possible solution, run the neo20-all.ahk explicitly with the AutoHotkeyU32.exe which can be found in the Autohotkey program directory. * In case you run a compiled neo20.exe, you should try to recompile the sources explicitly with the 32-bit binaries (Unicode 32-bit.bin). * Also make sure sure that you do not run the ASCII version of Autohotkey (AutoHotkeyA32.exe), which is not supported either. * Better yet, just fetch the latest version of the repo and recompile *(see @qwertfisch comment below)*
Owner

All NeoVars executables (for Neo, Bone and NeoQwertz) are compiled with Unicode 32-bit at least since faf4144d25.

All NeoVars executables (for Neo, Bone and NeoQwertz) are compiled with Unicode 32-bit at least since https://git.neo-layout.org/neo/neo-layout/commit/faf4144d25cb8ff02aced0b21be93412cf588373.

@qwertfisch Yes but @Polyhistorian compiled his own binaries [edit] at least 9 month ago, before that commit happened [/edit] and/or seems to run from a local copy of the repo, and the 64-bit issue might not be documented anywhere(?)

Neo-Vars: Newest from repo, built with the included scripts into an exe. Though the bug also persists with the loose scripts.

@qwertfisch Yes but @Polyhistorian compiled his own binaries [edit] *at least 9 month ago, before that commit happened* [/edit] and/or seems to run from a local copy of the repo, and the 64-bit issue might not be documented anywhere(?) > Neo-Vars: Newest from repo, built with the included scripts into an exe. Though the bug also persists with the loose scripts.
Owner

Yes, the issue is older than the insight about U32/U64. I remember an IRC discussion with Polyhistorian around that time.

Documentation about the issue is not explicit, only if you follow the build script from mentioned commit. I will add a section in documentation for developers – this page already needs complete rework.

Yes, the issue is older than the insight about U32/U64. I remember an IRC discussion with Polyhistorian around that time. Documentation about the issue is not explicit, only if you follow the build script from mentioned commit. I will add a section in [documentation for developers](https://neo-layout.org/Entwicklung/Treiber-KnowHow-NeoVars/) – this page already needs complete rework.

Ok, thank you for the info @qwertfisch. I have also updated my reply above.

As a side node: The reason why I replied here is because I stumbled into the same issue lately. With your commit, I did not find an underlying reason for making 32bit mandatory, but what I found recently is that adding 64bit support hinges on fixing a few DllCalls for sending unicode characters and also some in the keyboard LEDs module. I'll try to update a fork of mine on this site for inspiration (unfortunately the fork deviated quite a bit so it won't be a direct patch for the upstream version).

Ok, thank you for the info @qwertfisch. I have also updated my reply above. As a side node: The reason why I replied here is because I stumbled into the same issue lately. With your commit, I did not find an underlying reason for making 32bit mandatory, but what I found recently is that adding 64bit support hinges on fixing a few DllCalls for sending unicode characters and also some in the keyboard LEDs module. I'll try to update a fork of mine on this site for inspiration (unfortunately the fork deviated quite a bit so it won't be a direct patch for the upstream version).
Sign in to join this conversation.
Loading…
There is no content yet.