Difference between revisions of "Virtual Keyboard"

From ScummVM :: Wiki
Jump to navigation Jump to search
(→‎Information for keyboard pack creators: Removed Reference to SVN and changed link to Git SCM)
 
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The ScummVM virtual keyboard hopes to provide a generalised implementation
== Introduction ==
of a virtual keyboard that will be usable on all backends that
 
require it.
'''Note:''' The virtual keyboard is being actively developed.
 
The ScummVM virtual keyboard provides a generalized implementation
of a virtual keyboard that will be usable on any backends that wish to make use of it.


== Introduction ==
== Design ==


The keyboard uses a 'queued events' model (so anything you type is queued and then passed back to the main event routine when the keyboard is closed) so it is best used for entering input rather then as part of the game input system (look towards the [[Keymapping_Improvements]] for systems for real time input).  
The 1st key design consideration people need to be aware of is that the keyboard is rendered onto our 16bit overlay. This means that the keyboard exists outside of the scope of the running engine (the engine is paused when the overlay/keyboard is shown).


This also means that the virtual keyboard (in it's default state) will ensure the engine is paused when it is active.
By virtue of this design consideration the keyboard uses a 'queued events' model (so anything you select is queued and then injected back to the main event routine when the keyboard is closed) so it is best used for entering strings (save names etc.) rather than as part of the game input system (look towards the [[Keymapping_Improvements]] for systems for real time input).  


The virtual keyboard requires a valid 'keyboard pack' in order to operate. The
The virtual keyboard requires a valid 'keyboard pack' in order to operate. The pack should consist of an XML configuration file and a collection of 24-bit bitmaps. It can come in the form of a compressed zip file or a directory.
pack should consist of an XML configuration file and a collection of 24-bit
bitmaps. It can come in the form of a compressed zip file or a directory.


In order to make pack creation quick and easy, the virtual keyboard display
In order to make pack creation quick and easy, the virtual keyboard display uses complete bitmap images. To go with each bitmap a HTML-style image map must be created that defines the clickable regions of the image. The primary purpose of the XML configuration file is to contain the image map definitions along with associated event definitions and additional data.
uses complete bitmap images. To go with each bitmap a HTML-style image map must
be created that defines the clickable regions of the image. The primary purpose
of the XML configuration file is to contain the image map definitions along
with associated event definitions and additional data.


As it is impractical to display all possible keys at the same time, the virtual
As it is impractical to display all possible keys at the same time, the virtual keyboard has the ability to switch to different modes when special keys are pressed. Also, the virtual keyboard has to work at a range of different resolutions. Therefore, the pack must provide a bitmap and configuration data for each mode at each resolution.
keyboard has the ability to switch to different modes when special keys are
pressed. Also, the virtual keyboard has to work at a range of different
resolutions. Therefore, the pack must provide a bitmap and configuration data
for each mode at each resolution.


== Key Features ==
== Key Features ==


* Automatic resolution selection - with re-loading on change of resolution
* ''Automatic resolution selection - with re-loading on change of resolution'' - Not sure this works.


* Preview display - to show the keys that have been pressed while keyboard is active
* Preview display - to show the keys that have been pressed while keyboard is active
Line 37: Line 30:
* Transparency support - using keycolor bitmaps  
* Transparency support - using keycolor bitmaps  


* Draggable
* Can be dragged around the screen


== Special Events ==
== Special Events ==
Line 61: Line 54:
<event name="name_of_image_map_rect" type="delete" />
<event name="name_of_image_map_rect" type="delete" />
</syntax>
</syntax>
* move_left - Move left one place in the stacked event queue.
* move_left - Move left one place in the stacked event list (moves the position in the display_area).
<syntax type="xml">
<syntax type="xml">
<event name="name_of_image_map_rect" type="move_left" />
<event name="name_of_image_map_rect" type="move_left" />
</syntax>
</syntax>
* move_right - Move right one place in the stacked event queue if your not at the most recent event.
* move_right - Move right one place in the stacked event list if you’re not at the last event (moves the position in the display_area).
<syntax type="xml">
<syntax type="xml">
<event name="name_of_image_map_rect" type="move_right" />
<event name="name_of_image_map_rect" type="move_right" />
Line 74: Line 67:
'''NOTE:''' It should be noted that the ''wise'' keyboard pack author uses SVG for there keyboard image(s) and just saves bitmaps at whatever resolutions are needed. It also makes it easy to add extra resolution images later if needed.
'''NOTE:''' It should be noted that the ''wise'' keyboard pack author uses SVG for there keyboard image(s) and just saves bitmaps at whatever resolutions are needed. It also makes it easy to add extra resolution images later if needed.


In order to create a keyboard pack there are three main stages:  
In order to create a keyboard pack these are the main stages:  


* Create the keyboard bitmaps.
* Create the keyboard bitmaps.
Line 80: Line 73:
* Writing the XML configuration file and incorporate the image maps.
* Writing the XML configuration file and incorporate the image maps.


You can view the files that make up our default keyboard pack in SVN [http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/backends/vkeybd/packs/vkeybd_default/ here]. This should serve as a useful reference in conjunction with the information below.  
You can view the files that make up our default keyboard pack in our source code repository [https://github.com/scummvm/scummvm/tree/master/backends/vkeybd/packs/vkeybd_default here]. This should serve as a useful reference in conjunction with the information below.  


=== Keyboard bitmaps ===
=== Keyboard bitmaps ===
Line 92: Line 85:
After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.
After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.


==== Tag descriptions ====
==== Tag Descriptions ====


'''''TODO: format table properly'''''
{| border="1" cellspacing="0"
{|  
! Tag  
| Tag  
! Description
| Description
! Required Attributes
| Required attributes
! Optional Attributes
| Optional attributes
! Children
| Children
|- style="background:AliceBlue; color:black"
|-  
! <keyboard>
| <keyboard>
| This is the required, root element of the file format.  
| This is the required, root element of the file format.  
| initial_mode: name of the mode the keyboard will show initially.
| '''initial_mode''': name of the mode the keyboard will show initially.
| v_align/h_align: where on the screen should the keyboard appear initially (defaults to bottom/center).
| '''v_align'''/'''h_align''': where on the screen should the keyboard appear initially (defaults to bottom/centre).
| mode
! <mode>
|-
|- style="background:Azure; color:black"
| <mode>  
! <mode>
| This tag encapsulates a single mode of the keyboard. Within are a number of  
| This tag encapsulates a single mode of the keyboard. Within are a number of  
layouts, which provide the specific implementation at different resolutions.
layouts, which provide the specific implementation at different resolutions.
| name: the name of the mode  
| '''name''': the name of the mode  
resolutions: list of the different layout resolutions  
'''resolutions''': list of the different layout resolutions  
| layout
|  
event
| '''<layout>''' and '''<event>'''
|- style="background:HoneyDew; color:black"
! <layout>
| These tags encapsulate an implementation of a mode at a particular resolution.
| '''resolution''': the screen resolution that this layout is designed for
'''bitmap''': filename of the 24-bit bitmap that will be used for this layout (with no path)
| '''transparent_color''': color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255).
'''display_font_color''': color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).
| '''<map>''': This describes the image map using the same format as HTML image maps.
|-
|-
| <event>
! <map>
| These tags describe the image map for a particular layout. It uses the same
format as HTML image maps. The only area shapes that are supported are
rectangles and polygons. The target attribute of each area should be the name
of an event for this mode (see <event> tag). They will usually be generated by
an external tool such as GIMP's Image Map plugin, and so will not be written
by hand, but for more information on HTML image map format see W3Schools pages on the [http://www.w3schools.com/TAGS/tag_map.asp <map>] and [http://www.w3schools.com/TAGS/tag_area.asp <area>] tags.
|
|
| '''<area>''': This is part of the image map using the same format as HTML image maps.
|- style="background:HoneyDew; color:black"
! <event>
| These tags describe a particular event that will be triggered by a mouse click
| These tags describe a particular event that will be triggered by a mouse click
on a particular area. The target attribute of each image map area should be the
on a particular area. The target attribute of each image map area should be the
same as an event's name.
same as an event's name.
| name: name of the event
| name: name of the event
type: key | modifier | switch_mode | submit | cancel | clear | delete | move_left | move_right - see VirtualKeyboard::EventType for explanation
type: '''key''' | '''modifier''' | '''switch_mode''' | '''submit''' | '''cancel''' | '''clear''' | '''delete''' | '''move_left''' | '''move_right'''


for key events
for key events
code / ascii: describe a key press in ScummVM KeyState format  
code / ASCII: describe a key press in ScummVM KeyState format  


for key and modifier events
for key and modifier events
Line 131: Line 142:
for switch_mode events
for switch_mode events
mode: name of the mode that should be switched to
mode: name of the mode that should be switched to
|
|
|-
| <layout>
| These tags encapsulate an implementation of a mode at a particular resolution.
| resolution: the screen resolution that this layout is designed for
bitmap: filename of the 24-bit bitmap that will be used for this layout
| transparent_color: color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255).
display_font_color: color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).
| map: this describes the image map using the same format as html image maps
|-
| <map>
| These tags describe the image map for a particular layout. It uses the same
format as HTML image maps. The only area shapes that are supported are
rectangles and polygons. The target attribute of each area should be the name
of an event for this mode (see <event> tag). They will usually be generated by
an external tool such as GIMP's Image Map plugin, and so will not be written
by hand, but for more information on HTML image map format see W3Schools pages on the [http://www.w3schools.com/TAGS/tag_map.asp <map>] and [http://www.w3schools.com/TAGS/tag_area.asp <area>] tags.
|
|
|
|
|
Line 208: Line 200:


== Information for users ==
== Information for users ==
 
The keyboard will pause the underlying game engine when in use.
'''''TODO'''''
All text/events that are going to be passed back to the game engine are shown in the preview area (display_area in the keyboard pack).
pauses system
preview area

Latest revision as of 21:11, 10 July 2011

Introduction

Note: The virtual keyboard is being actively developed.

The ScummVM virtual keyboard provides a generalized implementation of a virtual keyboard that will be usable on any backends that wish to make use of it.

Design

The 1st key design consideration people need to be aware of is that the keyboard is rendered onto our 16bit overlay. This means that the keyboard exists outside of the scope of the running engine (the engine is paused when the overlay/keyboard is shown).

By virtue of this design consideration the keyboard uses a 'queued events' model (so anything you select is queued and then injected back to the main event routine when the keyboard is closed) so it is best used for entering strings (save names etc.) rather than as part of the game input system (look towards the Keymapping_Improvements for systems for real time input).

The virtual keyboard requires a valid 'keyboard pack' in order to operate. The pack should consist of an XML configuration file and a collection of 24-bit bitmaps. It can come in the form of a compressed zip file or a directory.

In order to make pack creation quick and easy, the virtual keyboard display uses complete bitmap images. To go with each bitmap a HTML-style image map must be created that defines the clickable regions of the image. The primary purpose of the XML configuration file is to contain the image map definitions along with associated event definitions and additional data.

As it is impractical to display all possible keys at the same time, the virtual keyboard has the ability to switch to different modes when special keys are pressed. Also, the virtual keyboard has to work at a range of different resolutions. Therefore, the pack must provide a bitmap and configuration data for each mode at each resolution.

Key Features

  • Automatic resolution selection - with re-loading on change of resolution - Not sure this works.
  • Preview display - to show the keys that have been pressed while keyboard is active
  • Modifier keys - allow Shift, Control, and Alt to be added to keystroke without using a separate mode
  • Special events - to submit or cancel the keyboard, switch mode, etc
  • Transparency support - using keycolor bitmaps
  • Can be dragged around the screen

Special Events

In addition to the mapping of keys and the keyboard modes you can also wire up a number of special events to image map areas.

The current 'special events' are as follows.

  • submit - Submit all the events queues (as shown in the display area) and close the keyboard.

<syntax type="xml"> <event name="name_of_image_map_rect" type="submit" /> </syntax>

  • cancel - Close the keyboard and cancel all queued events.

<syntax type="xml"> <event name="name_of_image_map_rect" type="cancel" /> </syntax>

  • clear - Clear all queued events.

<syntax type="xml"> <event name="name_of_image_map_rect" type="clear" /> </syntax>

  • delete - Delete the last event from the queue (think backspace).

<syntax type="xml"> <event name="name_of_image_map_rect" type="delete" /> </syntax>

  • move_left - Move left one place in the stacked event list (moves the position in the display_area).

<syntax type="xml"> <event name="name_of_image_map_rect" type="move_left" /> </syntax>

  • move_right - Move right one place in the stacked event list if you’re not at the last event (moves the position in the display_area).

<syntax type="xml"> <event name="name_of_image_map_rect" type="move_right" /> </syntax>

Information for keyboard pack creators

NOTE: It should be noted that the wise keyboard pack author uses SVG for there keyboard image(s) and just saves bitmaps at whatever resolutions are needed. It also makes it easy to add extra resolution images later if needed.

In order to create a keyboard pack these are the main stages:

  • Create the keyboard bitmaps.
  • Construct the HTML image maps (use something like GIMP).
  • Writing the XML configuration file and incorporate the image maps.

You can view the files that make up our default keyboard pack in our source code repository here. This should serve as a useful reference in conjunction with the information below.

Keyboard bitmaps

ScummVM's image manager can only decode 24-bit BMP images, so make sure you save your images in this format. You need to reserve a particular color value to be used for transparency. By default the color used for transparency is set to be #FF00FF, but if you want to use a different color this can be specified in the configuration file.

XML configuration file

The first step in creating the XML configuration file is to create the HTML imagemap data. The easiest way to do this is by using an standard image map creation tool such as GIMP's Image Map plugin (find it under Filters -> Web -> Image Map...). You will need to do this for each bitmap, ensuring that you define areas for all the clickable areas (and the display box). Also ensure that you only use rectangle or polygon map areas, currently circle areas are not supported.

After creating all the image maps you should put the exported HTML for all the bitmaps into a single XML file. Then you can start adding the rest of the configuration XML around it. The rest of this section provides descriptions of the tags and example complete file.

Tag Descriptions

Tag Description Required Attributes Optional Attributes Children
<keyboard> This is the required, root element of the file format. initial_mode: name of the mode the keyboard will show initially. v_align/h_align: where on the screen should the keyboard appear initially (defaults to bottom/centre). <mode>
<mode> This tag encapsulates a single mode of the keyboard. Within are a number of

layouts, which provide the specific implementation at different resolutions.

name: the name of the mode

resolutions: list of the different layout resolutions

<layout> and <event>
<layout> These tags encapsulate an implementation of a mode at a particular resolution. resolution: the screen resolution that this layout is designed for

bitmap: filename of the 24-bit bitmap that will be used for this layout (with no path)

transparent_color: color in r,g,b format that will be used for keycolor transparency (defaults to (255,0,255).

display_font_color: color in r,g,b format that will be used for the text of the keyboard display (defaults to (0,0,0).

<map>: This describes the image map using the same format as HTML image maps.
<map> These tags describe the image map for a particular layout. It uses the same

format as HTML image maps. The only area shapes that are supported are rectangles and polygons. The target attribute of each area should be the name of an event for this mode (see <event> tag). They will usually be generated by an external tool such as GIMP's Image Map plugin, and so will not be written by hand, but for more information on HTML image map format see W3Schools pages on the <map> and <area> tags.

<area>: This is part of the image map using the same format as HTML image maps.
<event> These tags describe a particular event that will be triggered by a mouse click

on a particular area. The target attribute of each image map area should be the same as an event's name.

name: name of the event

type: key | modifier | switch_mode | submit | cancel | clear | delete | move_left | move_right

for key events code / ASCII: describe a key press in ScummVM KeyState format

for key and modifier events modifiers: modifier keystate as comma-separated list of shift, ctrl and/or alt.

for switch_mode events mode: name of the mode that should be switched to

Example configuration file

<syntax type="xml"> <keyboard modes="normal,caps" initial_mode="normal" v_align="bottom" h_align="center">

 <mode name="normal" resolutions="640x400,320x200">
   <layout resolution="640x400" bitmap="normal_640x400.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="q" />
       <area shape="poly" coords="105,50,107,48,134,48,136,50,136,77,134,79,107,79,105,77" target="w" />
       <area shape="poly" coords="146,50,148,48,174,48,176,50,176,77,174,79,148,79,146,77" target="e" />
       ...
       <area shape="poly" coords="11,89,12,88,69,88,70,89,70,116,69,117,12,117,11,116" target="caps" />
     </map>
   </layout>
   <layout resolution="320x200" bitmap="normal_320x200.bmp" transparent_color="255,0,255">
     ...
   </layout>
   <event name="a" type="key" code="97" ascii="97" modifiers="" />
   <event name="b" type="key" code="98" ascii="98" modifiers="" />
   <event name="c" type="key" code="99" ascii="99" modifiers="" />
   ...
   <event name="caps" type="switch_mode" mode="caps" />
 </mode>
 <mode name="caps" resolutions="640x400">
   <layout resolution="640x400" bitmap="caps_640x480.bmp" transparent_color="255,0,255">
     <map>
       <area shape="poly" coords="65,50,67,48,94,48,96,50,96,77,94,79,67,79,65,77" target="Q" />
       ...
     </map>
   </layout>
   <event name="A" type="key" code="97" ascii="65" modifiers="shift" />
   <event name="B" type="key" code="98" ascii="66" modifiers="shift" />
   <event name="C" type="key" code="99" ascii="67" modifiers="shift" />
   ...
 </mode>

</keyboard> </syntax>

Information for porters

By default the virtual keyboard code is disabled from a ScummVM build. In order to enable it use the --enable-vkeybd flag when running the configure script.

After you have enabled the virtual keyboard in a build the loading code must be provided with a path to find the keyboard pack(s), and the name of the keyboard pack to use.

For the path the loading code uses the value of the 'vkeybdpath' config file variable if it is defined. If it is not defined it will revert to the 'extrapath' variable. Likewise, if that is not defined it default to the current directory.

For the name of the keyboard pack, the loading code uses the 'vkeybd_pack_name' variable if defined, if not it defaults to using 'vkeybd_default' for the pack name.

Once the keyboard pack path and name are defined the loading code looks for a valid keyboard pack. First it looks for an uncompressed pack by looking for a file with the name [packname].xml in the keyboard pack directory. If it does not find this it will look for a compressed pack by looking for a file with the name [packname].zip.

To use the keyboard within your port ensure that your choice of keyboard pack has a suitable resolution (<= your overlay resolution) and that you have wired up something in your event handling code to generate Common::KEYCODE_F7.

Information for users

The keyboard will pause the underlying game engine when in use. All text/events that are going to be passed back to the game engine are shown in the preview area (display_area in the keyboard pack).