Feature #14713

Discuss and evaluate feedback coming from the Florence removal

Added by intrigeri 2017-09-24 08:50:32 . Updated 2018-12-02 22:28:35 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Hardware support
Target version:
Start date:
2017-10-03
Due date:
% Done:

100%

Feature Branch:
Type of work:
Test
Blueprint:

Starter:
Affected tool:
On-screen keyboard
Deliverable for:

Description

Pablo wrote somewhere else (Bug #14675#note-16):

> But I think the important points for accessibility are:
>
> # The user’s ability to keep the keyboard visible at any time not only in programs and specific times.
> # Ability to place and move the keyboard at any location on the screen as in Florence using a button.
> # Ability to adjust the keyboard size.
> # (Dispensable point but I think important) a beautiful keyboard not with old-looking buttons.
>
> As a user I prefer option 1. (I’m sorry for your sadness Intrigeri)
>
> Follow my personal reasons for use. I few times use the tails on a computer 2 in 1 (HP Pavillion), ie sometimes I double the screen and use the tails as a “tablet”, something that brings a bad experience But that worked since using with mouse and touch, and then with the end of the keyboard Florence I can not have any autonomy, for example I had before, trying to enter an address in the browser or a simple search, did this using the virtual keyboard (Florence) and the mouse , and just now I can’t do it anymore because the keyboard doesn’t show up at all the locations I want, it appeared to me just in the text editors, well another change and now I can no longer put the keyboard in place of the screen where I want , it (when it pops up) just gets stuck down by occupying the entire screen at the bottom, in the spaces of the sidelines totally useless and paralyzed, well the improvement was design, because on the keyboard Florence, was in a very “old” way, anyway, the functionality of having the virtual keyboard in Time and place of the screen I want was important, because so I could use, although in a rustic way to combine Florence-mouse-touch of the tails, the keyboard property and not always available does not seem to be accessible.

Subtasks

Bug #14763: It should be possible to turn off the on-screen keyboard if desired Resolved intrigeri

0

Related issues

Related to Tails - Feature #8281: Consider replacing Florence with GNOME's own on-screen keyboard Resolved 2014-11-20
Related to Tails - Feature #16182: Evaluate on-screen keyboard on Buster Resolved 2018-12-02

History

#1 Updated by intrigeri 2017-09-24 08:51:06

  • Description updated

#2 Updated by intrigeri 2017-09-24 08:51:15

  • related to Feature #8281: Consider replacing Florence with GNOME's own on-screen keyboard added

#3 Updated by intrigeri 2017-09-24 09:04:05

  • Subject changed from Discuss and evaluate feedback coming from the Florence removal. to Discuss and evaluate feedback coming from the Florence removal

#4 Updated by intrigeri 2017-09-24 09:27:12

  • Assignee changed from intrigeri to pablonatalino
  • QA Check set to Info Needed

Hi Pablo!

First of all I would like to clear up some potential confusion: our intent for Tails 3.2 was not to simply drop Florence, but to replace it with GNOME’s own on-screen keyboard. Sadly, this was broken in 3.2~rc1 that effectively shipped with no usable screen keyboard at all. So I’m not sure if your comment is about what 3.2 will look like, or about what the buggy 3.2~rc1 shipped, or about Tails 3.1 (some of the issues you’re reporting do occur on 3.1 but are supposed to be fixed now). Can you please clarify? It would be very helpful if you could test again (on that 2-in-1 computer) with a more recent ISO that behaves like 3.2 will: https://nightly.tails.boum.org/build_Tails_ISO_testing/lastSuccessful/archive/build-artifacts/

Note, in passing, that Florence and its integration into Tails were very buggy

Now I’ll reply to specific bits:

>> But I think the important points for accessibility are:
>>
>> The user’s ability to keep the keyboard visible at any time not only in programs and specific times.

Now you make me very curious: may I ask why you would want to display the on-screen keyboard even when you don’t need it? I’ve not seen such behavior on any touch device / operating system so far. Have you?

>> Ability to place and move the keyboard at any location on the screen as in Florence using a button.
>> Ability to adjust the keyboard size.

From both a UX and code perspective, I think these are the wrong solutions to a couple of very real problems. I mean:

  • very real problems about the GNOME on-screen keyboard:
    • it sometimes hides the input field one wants to use it for; that’s clearly a bug and it’s tracked upstream (it was actually reported by GNOME’s lead designer / UX person); I have good hope that there’s progress made in this area as the GNOME project is getting more and more interested in supporting touch devices, e.g. https://wiki.gnome.org/Design/OS/ScreenKeyboard. It seems that the affected code is in JavaScript, which makes it easier to contribute than if it were in C by the way.
    • the default size of the keyboard might not be correctly and automatically adapted on some devices; I’ve not found any bug report about this, but provided enough information (hardware details, screenshot) I’d be happy to report it upstream. Note that resizing was broken with Florence anyway (Bug #12423).
  • wrong solutions: I think everyone’s time will be better spent in fixing the root causes of the aforementioned problems, and making it work fine automatically, than in implementing workarounds that require the user to manually manage the size and placement of the on-screen keyboard. I suspect that fixing the root causes does not actually require more work.

>> Dispensable point but I think important) a beautiful keyboard not with old-looking buttons.

I totally agree that nice looking software is more pleasant to use; that’s part of UX :)

Now, “nice looking” is somewhat subjective (unless user testing data is provided of course). This is clearly not my area of expertise, all I can say is that the current design looks OK to me, and is rather well integrated with the rest of the GNOME Shell interface in my (humble!) opinion. https://wiki.gnome.org/Design/OS/ScreenKeyboard suggests that the current GNOME on-screen keyboard looks very much like the Windows one. I suspect that Microsoft and/or GNOME have done actual research to validate their choice. I’m sure the GNOME designers would be happy to get feedback (and possibly data) about this, so if you want to get involved in making that screen keyboard more beautiful, please let me know and I’ll put you in touch with them :)

>> Follow my personal reasons for use. I few times use the tails on a computer 2 in 1 (HP Pavillion), ie sometimes I double the screen and use the tails as a “tablet”,

Nice. I’ve added exactly the same setup to my daily (non-work) flow yesterday. I’m using regular Debian unstable instead of Tails (that’s an ARM machine that Tails can’t run on), but the desktop is essentially the same. This experience will help me relate to this topic from a user-centric perspective in the future :)

> something that brings a bad experience

That is? Anything else than the 3 issues listed below?

> But that worked since using with mouse and touch, and then with the end of the keyboard Florence I can not have any autonomy, for example I had before, trying to enter an address in the browser or a simple search, did this using the virtual keyboard (Florence) and the mouse , and just now I can’t do it anymore because the keyboard doesn’t show up at all the locations I want, it appeared to me just in the text editors,

This specific problem (that occurred on Tails 3.1) is now fixed in the ISO image I’ve pointed you to above, and will be fixed in Tails 3.2. During my testing (Feature #8281) I could not find any text input widget that would not cause the screen keyboard to pop up. Nobody answered my call for testing so I don’t know if there are still issues. I would love if you could test that newer ISO and check if there are still such problems :)

> well another change and now I can no longer put the keyboard in place of the screen where I want , it (when it pops up) just gets stuck down by occupying the entire screen at the bottom, in the spaces of the sidelines totally useless and paralyzed, well the improvement was design, because on the keyboard Florence, was in a very “old” way, anyway, the functionality of having the virtual keyboard in Time and place of the screen I want was important, because so I could use, although in a rustic way to combine Florence-mouse-touch of the tails, the keyboard property and not always available does not seem to be accessible.

See my comment above on the size and placement topic.

#5 Updated by pablonatalino 2017-09-26 16:18:55

Hi intrigeri! I have already read the comment, already installed 3.2 (more recent ISO), and I will do as soon as possible the test and comment with property.

#6 Updated by anonym 2017-09-26 18:06:47

Comment Feature #10263#note-11 might perhaps belong better here(?).

#7 Updated by pablonatalino 2017-09-29 17:10:03

Ok, come on, I’ll tell you in detail for better and exact compression (I had to use an online translator)!

Well, I don’t talk about a common touch on a notebook, where the differential and only the screen with a touch, I own a HP Pavilion x360 touch Notebook. This notebook has the ability to position the screen at 360 degrees. This is an incredible capacity (at least for me), and in various positions. It follows here image so you have more real idea than I speak http://www.www8-hp.com/in/en/images/gallery-2_tcm_188_1580174.jpg. If I rotate the 360 screen The notebook is equal to a common tablet, up to 90 ~ 170 is normal as a notebook, with the keyboard enabled. But from 180 º until 270 º is similar to the notebook, but with the screen “out” and keyboard down, the keyboard is disabled if I turn it on from 180 º..

A countless possibilities in which I use the 360 º. But what I need most is the next. I use this notebook, in bed, in positions totally laying down, three things to consider themselves as important.

1. Security, when I’m going to sleep, or just to relax my body using the tails, I do two basic things when I use the Lying notebook: Browse the Internet Only “read” websites, or a PDF or book, or the function I do very much is to watch videos. I always do this watch videos, but have chances of sleeping and not removing the notebook from the bed and not putting it on the floor near the bed, this is somewhat insecure because if I sleep I can break or let the computer fall from the bed, or lie on it at last And “Dangerous to fall asleep with a computer in bed”. So what I do is, I haven’t broken anything with this behaviour, I usually start with the computer on top of my breast in the position of 270 degrees, this doesn’t always work in the tails because the keyboard gets in touch with my chest and if it’s not with a thick shirt I can’t disable the physical keyboard, this happens if it’s in a tttttttttssssss text field. And then when I’m in some drowsiness I put the notebook beside my pillow in the position of 270 º, and then I can sleep more peacefully, watching some video for example.

2. Health. Well I am very afraid of getting sore necks and especially with the eyes in an uncomfortable position, because I almost never know when I will be able to sleep, when I use the computer in bed, and in this case and completely lying down, the worst thing and having to enter a term to do some Video research. It bothers and have flexed arms trying to use the keyboard and or touch, the touch also does not work in good shape for those who are lying down. So I always use the wireless mouse and all my comas to pass a page to change or pause a video or a simple search to find a video, and using a mouse with the dongle. And that’s what the keyboard Florence used for.

3. Usability. Not to do everything always on the Florence keyboard, but it works for what I do in a limited way, read or watch videos, help in many things. It also has no “aceleromenter”. The tails do not have the same integration with the touch

I do this also in Windows 10 in tablet mode (I’m tempted everything to get out of Windows), and amazing but I can do everything, and it seems everything was thought of by Windows and also by HP. There is also a functionality in Windows, that the “light sundowning” , according to the description of this functionality in Windows 10, the blue light present on the computer screen inhibits sleep, and then I can activate the night light that causes the computer’s screen colors to become “hot” and then encourages the emergence of sleep. As I also have a wireless keyboard (Bluetooth), I get many things completely lying down, and most importantly with stretched arms, the wireless keyboard has lighting and when I do that it stays on my waist, so in Windows I can type and until writing texts Usually with the notebook a ~ 270 º side of the pillow.

#8 Updated by goupille 2017-10-03 12:39:00

I add some users feedback we received :

One user complained about the lack of ability to use Gnome on-screen keyboard without clicking (apparently it was possible with Florence)

and another user reported Feature #14713

#9 Updated by intrigeri 2017-10-17 11:05:26 

tkiope: hello
tkiope: i cant use tails now because screen keyboard dont have all the keys any more
tkiope: p.s. disability person
intrigeri: tkiope: what keys are missing? have you tried long-press? (e.g. long-press on "e" gives you all kinds of accented e's)
tkiope: arrows and function keys
tkiope: also shift key cant be put on hold
tkiope: I mentioned two, the third on is the keyboard is not floating so cant click on the portion of any window that comes behind it so I need to minimise the keyboard first
tkiope: but even minimise sometimes doest solve this isuue as it pops up again when clicking on the portion that was previously hidden, then back to same issue
tkiope: proposed solution:floating feature or a way not to force keyboard to always be infront of everything on screen
tkiope: fourth is probably a mixed issue: no way to change layout to on screen keyboard
tkiope: fifth: no on screen keyboard neither on tails greeter nor to fill in any passwords after loging in to tails session. examples: entering wifi password
tkiope: that is all for now. maybe there is a newer keyboard version for gnome with more features?
tkiope: thanks for help.. hope to see a fast resolution otherwise tails for me is much less usable

#10 Updated by intrigeri 2017-10-17 11:15:23

  • Assignee changed from pablonatalino to intrigeri
  • QA Check deleted (Info Needed)

#11 Updated by intrigeri 2017-10-29 07:54:47

#12 Updated by intrigeri 2017-11-10 17:12:07

Hi goupille!

goupille wrote:
> One user complained about the lack of ability to use Gnome on-screen keyboard without clicking (apparently it was possible with Florence)

I have no idea what “without clicking” means in this context. Can you please clarify? Clicking with what, clicking where?

#13 Updated by intrigeri 2017-11-10 17:48:13

Processing tkioepe’s report:

  • no arrows keys: I see that the Windows 8 on-screen keyboard has left/right arrows (no up/down); the touchscreen oriented iOS and Android ones have none; the way I understand the design, one is supposed to use the mouse for most things where the arrow keys would be useful; but this does not work everywhere, e.g. I don’t know how to browse the shell history in a Terminal; if there are other (perhaps less geeky?) use cases affected by this problem, I’m open to starting a discussion upstream about it
  • no function keys: it’s stated as a non-goal upstream; I would hope we don’t ship anything user-facing that can be used only with Fn keys and not usable with the mouse, and in terms of a11y the on-screen keyboard is meant for people who can use a mouse but not a keyboard, so I don’t think it’s a big problem and I won’t dare asking upstream to revisit their design (I would not be able to explain them why it’s needed). Sorry if I missed something!
  • no caps lock emulation: the upstream tentative design says it should be triggered by “pressing and holding the shift key”, but apparently this is not implemented yet; so I’ve reported it upstream
  • on-screen keyboard can hide windows: we knew that when discussing Feature #8281, tracked upstream and mentioned on the upstream design page already
  • no way to change layout: we knew that when discussing Feature #8281, already tracked upstream; Alan reported some progress about this on Feature #8281, no idea what’s the status, I’ll let Alan report on the ticket we have about monitoring this upstream
  • “no on screen keyboard neither on tails greeter nor to fill in any passwords after loging in to tails session. examples: entering wifi password”: it works fine for me in the Greeter and when we tested it for Feature #8281 it also worked fine in GNOME Shell’s password entry fields; but it takes a while to start the first time (which we document), which probably explains tkioepe’s experience. Not sure what we can do better about it :/

#14 Updated by intrigeri 2017-11-10 17:51:24

intrigeri wrote:
> I have no idea what “without clicking” means in this context. Can you please clarify? Clicking with what, clicking where?

I think I got it: did you mean “hover to click” (on keys)? It’s listed as a desired feature on the upstream design page.

#15 Updated by intrigeri 2017-11-10 17:53:04

Note to myself: last piece of feedback I should process is Pablo’s Feature #14713#note-7.

#16 Updated by intrigeri 2017-11-10 17:58:17

  • Target version changed from Tails_3.3 to Tails_3.5

Hi Pablo!

pablonatalino wrote:
> Ok, come on, I’ll tell you in detail for better and exact compression (I had to use an online translator)!

I’m confused: you did not answer my questions — please do :) Thanks for describing the convertible laptop use case: it’s totally valid, and actually it is now part of my daily routine so I know how nice it is. But I could not find anything about the new on-screen keyboard in this lengthy description. So I’m not sure what I’m supposed to do about it, especially on this ticket.

#17 Updated by intrigeri 2017-11-10 17:58:51

  • Assignee changed from intrigeri to pablonatalino
  • QA Check set to Info Needed

#18 Updated by goupille 2017-11-10 19:00:19

intrigeri wrote:
> did you mean “hover to click” (on keys)? It’s listed as a desired feature on the upstream design page.

it is exactly what I meant, sorry, I forgot about the existence of the word ‘hover’ :)

#19 Updated by intrigeri 2017-11-18 11:46:52

I’ve shared the feedback we’ve received with the GNOME design team.

#20 Updated by intrigeri 2018-01-01 16:39:53

  • blocked by deleted (Feature #13244: Core work 2017Q4: Foundations Team)

#21 Updated by intrigeri 2018-01-01 16:39:57

#22 Updated by anonym 2018-01-23 19:52:52

  • Target version changed from Tails_3.5 to Tails_3.6

#23 Updated by intrigeri 2018-02-20 08:50:35

intrigeri wrote:
> Hi Pablo!
>
> pablonatalino wrote:
> > Ok, come on, I’ll tell you in detail for better and exact compression (I had to use an online translator)!
>
> I’m confused: you did not answer my questions — please do :)

Ping?

#24 Updated by bertagaz 2018-03-14 11:32:26

  • Target version changed from Tails_3.6 to Tails_3.7

#25 Updated by intrigeri 2018-03-14 15:39:27

  • Assignee changed from pablonatalino to intrigeri
  • Target version changed from Tails_3.7 to Tails_4.0
  • QA Check deleted (Info Needed)
  • Type of work changed from Discuss to Test

intrigeri wrote:
> * on-screen keyboard can hide windows: we knew that when discussing Feature #8281, tracked upstream and mentioned on the upstream design page already

The GNOME 3.28 Release Notes state that now “the view is shifted to ensure that the text area is visible while typing” so this won’t be a problem anymore in Tails 4.0, based on Buster :)

> * no way to change layout: we knew that when discussing Feature #8281, already tracked upstream; Alan reported some progress about this on Feature #8281, no idea what’s the status, I’ll let Alan report on the ticket we have about monitoring this upstream

In GNOME 3.28 “a variety of layouts are supported for different locales”.

I’ll stop waiting for pablonatalino feedback and am recycling this ticket to evaluate this rewritten on-screen keyboard in GNOME 3.28. My main goals will be:

  • check that the aforementioned problems are indeed fixed
  • check that there’s no regression

#26 Updated by intrigeri 2018-03-27 09:16:20

  • blocked by deleted (Feature #13245: Core work 2018Q1: Foundations Team)

#27 Updated by intrigeri 2018-12-02 22:28:11

  • related to Feature #16182: Evaluate on-screen keyboard on Buster added

#28 Updated by intrigeri 2018-12-02 22:28:35

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • Target version deleted (Tails_4.0)

Filed Feature #16182 for the next steps.