OSDN Git Service

Fixes and improvements related to touch screen gestures.
authorMichael Curran <mick@nvaccess.org>
Tue, 15 Dec 2015 05:58:57 +0000 (15:58 +1000)
committerJames Teh <jamie@nvaccess.org>
Fri, 22 Jan 2016 02:55:03 +0000 (12:55 +1000)
* All taps that overlap in time are correctly merged into one multi-finger tap. Previously tap A and B would be merged only if B started after A's start but before its end.
* Taps are only merged into plural (double, tripple, quadruple) taps if there are no other unknown trackers at the time. This stops a tap quickly followed by a tripple tap for example, from incorrectly turning into a quadruple tap.
* Plural tap and holds (e.g. double tap and hold) are now detected.
* Hover downs are no longer fired when part of a tapAndHold.
* Hovers are no longer reported in input help.
* MultitouchTracker objects now contain a childTrackers property which contains the multiTouchTrackers it is made up of (I.e. 2 finger double tap contains 2 2-finger taps. The 2-finger taps themselves contain 2 taps).
* MultiTouchTracker objects now also contain a rawSingleTouchTracker property, if this tracker is the result of one single finger doing a tap, flick or hover. The SingleTouchTracker allows access to the underlying Operating System assigned ID for the finger and whether or not the finger is still in contact at the current time.
* Added doc strings to MultiTouchTracker and SingleTouchTracker classes explaining all available properties
* Provide __slots__ on MultiTouchTracker and SingleTouchTracker classes to decrease their size.
* The Touch input Gesture class's doc string now explains all possible actions that can occur
* Touch input gestures now have x and y properties, removing the need to access the tracker in very simple cases.
* Gesturs now contain a preheldTracker property, which is a multitouchTracker object representing the other held fingers while this action was being performed.
* Gesture identifier IDs are now output in Python set order ensuring correct comparisons, allowing gestures such as "1finger_hold+hover" to be correctly bound.
* A generalized identifier with finger count removed is now included for holds (e.g. hold+hover for 1finger_hold+hover).

Fixes #5652.

source/globalCommands.py
source/inputCore.py
source/touchHandler.py
source/touchTracker.py
user_docs/en/changes.t2t

index 94d4502..796e7eb 100755 (executable)
@@ -1816,13 +1816,13 @@ class GlobalCommands(ScriptableObject):
 \r
 \r
        def script_touch_newExplore(self,gesture):\r
-               touchHandler.handler.screenExplorer.moveTo(gesture.tracker.x,gesture.tracker.y,new=True)\r
+               touchHandler.handler.screenExplorer.moveTo(gesture.x,gesture.y,new=True)\r
        # Translators: Input help mode message for a touchscreen gesture.\r
        script_touch_newExplore.__doc__=_("Reports the object and content directly under your finger")\r
        script_touch_newExplore.category=SCRCAT_TOUCH\r
 \r
        def script_touch_explore(self,gesture):\r
-               touchHandler.handler.screenExplorer.moveTo(gesture.tracker.x,gesture.tracker.y)\r
+               touchHandler.handler.screenExplorer.moveTo(gesture.x,gesture.y)\r
        # Translators: Input help mode message for a touchscreen gesture.\r
        script_touch_explore.__doc__=_("Reports the new object or content under your finger if different to where your finger was last")\r
        script_touch_explore.category=SCRCAT_TOUCH\r
index f8e4f28..962600c 100644 (file)
@@ -58,6 +58,10 @@ class InputGesture(baseObject.AutoPropertyObject):
        #: @type: bool\r
        bypassInputHelp=False\r
 \r
+       #: Indicates that this gesture should be reported in Input help mode. This would only be false for floodding Gestures like touch screen hovers.\r
+       #: @type: bool\r
+       reportInInputHelp=True\r
+\r
        def _get_identifiers(self):\r
                """The identifier(s) which will be used in input gesture maps to represent this gesture.\r
                These identifiers will be looked up in order until a match is found.\r
@@ -460,7 +464,7 @@ class InputManager(baseObject.AutoPropertyObject):
 \r
        def _inputHelpCaptor(self, gesture):\r
                bypass = gesture.bypassInputHelp or getattr(gesture.script, "bypassInputHelp", False)\r
-               queueHandler.queueFunction(queueHandler.eventQueue, self._handleInputHelp, gesture, onlyLog=bypass)\r
+               queueHandler.queueFunction(queueHandler.eventQueue, self._handleInputHelp, gesture, onlyLog=bypass or not gesture.reportInInputHelp)\r
                return bypass\r
 \r
        def _handleInputHelp(self, gesture, onlyLog=False):\r
index 5cf3c79..5288e5d 100644 (file)
@@ -99,6 +99,24 @@ touchWindow=None
 touchThread=None\r
 \r
 class TouchInputGesture(inputCore.InputGesture):\r
+       """\r
+       Represents a gesture performed on a touch screen.\r
+       Possible actions are:\r
+       * Tap: a finger touches the screen only for a very short amount of time.\r
+       * Flick{Left|Right|Up|Down}: a finger swipes the screen in a particular direction.\r
+       * Tap and hold: a finger taps the screen but then again touches the screen, this time remaining held.\r
+       * Hover down: A finger touches the screen long enough for the gesture to not be a tap, and it is also not already part of a tap and hold. \r
+       * Hover: a finger is still touching the screen, and may be moving around. Only the most recent finger to be hovering causes these gestures.\r
+       * Hover up: a finger that was classed as a hover, releases contact with the screen.\r
+       All actions accept for Hover down, Hover and Hover up, can be made up of multiple fingers. It is possible to have things such as a 3-finger tap, or a 2-finger Tap and Hold, or a 4 finger Flick right.\r
+       Taps maybe pluralized (I.e. a tap very quickly followed by another tap of the same number of fingers will be represented by a double tap, rather than two separate taps). Currently double, tripple and quadruple plural taps are detected.\r
+       Tap and holds can be pluralized also (E.g. a double tap and hold means that there were two taps before the hold).\r
+       Actions also communicate if other fingers are currently held while performing the action. E.g. a hold+tap is when a finger touches the screen long enough to become a hover, and a tap with another finger is performed, while the first finger remains on the screen. Holds themselves also can be made of multiple fingers.\r
+       Based on all of this, gestures could be as complicated as a 5-finger hold + 5-finger quadruple tap and hold.\r
+       To find out the generalized point on the screen at which the gesture was performed, use this gesture's x and y properties.\r
+       If low-level information  about the fingers and sub-gestures making up this gesture is required, the gesture's tracker and preheldTracker properties can be accessed. \r
+       See touchHandler.MultitouchTracker for definitions of the available properties.\r
+       """\r
 \r
        counterNames=["single","double","tripple","quodruple"]\r
 \r
@@ -120,30 +138,43 @@ class TouchInputGesture(inputCore.InputGesture):
                if self.tracker.action in (touchTracker.action_hover,touchTracker.action_hoverUp): return None\r
                return super(TouchInputGesture,self).speechEffectWhenExecuted\r
 \r
-       def __init__(self,tracker,mode):\r
+       def _get_reportInInputHelp(self):\r
+               return self.tracker.action!=touchTracker.action_hover\r
+\r
+       def __init__(self,preheldTracker,tracker,mode):\r
                super(TouchInputGesture,self).__init__()\r
                self.tracker=tracker\r
+               self.preheldTracker=preheldTracker\r
                self.mode=mode\r
+               self.x=tracker.x\r
+               self.y=tracker.y\r
 \r
        def _get__rawIdentifiers(self):\r
-               ID=""\r
-               if self.tracker.numHeldFingers>0:\r
-                       ID+="%dfinger_hold+"%self.tracker.numHeldFingers\r
-               if self.tracker.numFingers>1:\r
-                       ID+="%dfinger_"%self.tracker.numFingers\r
-               if self.tracker.actionCount>1:\r
-                       ID+="%s_"%self.counterNames[min(self.tracker.actionCount,4)-1]\r
-               ID+=self.tracker.action\r
                IDs=[]\r
-               IDs.append("TS(%s):%s"%(self.mode,ID))\r
-               IDs.append("ts:%s"%ID)\r
+               for includeHeldFingers in ([True,False] if self.preheldTracker else [False]):\r
+                       ID=""\r
+                       if self.preheldTracker:\r
+                               ID+=("%dfinger_hold+"%self.preheldTracker.numFingers) if includeHeldFingers else "hold+"\r
+                       if self.tracker.numFingers>1:\r
+                               ID+="%dfinger_"%self.tracker.numFingers\r
+                       if self.tracker.actionCount>1:\r
+                               ID+="%s_"%self.counterNames[min(self.tracker.actionCount,4)-1]\r
+                       ID+=self.tracker.action\r
+                       IDs.append("TS(%s):%s"%(self.mode,ID))\r
+                       IDs.append("ts:%s"%ID)\r
                return IDs\r
 \r
        def _get_logIdentifier(self):\r
                return self._rawIdentifiers[0]\r
 \r
        def _get_identifiers(self):\r
-               return [x.lower() for x in self._rawIdentifiers] \r
+               identifiers=[]\r
+               for identifier in self._rawIdentifiers:\r
+                       t,i=identifier.split(':')\r
+                       # Force the ID in to Python set order so they are always comparable\r
+                       i="+".join(set(i.split("+")))\r
+                       identifiers.append("%s:%s"%(t.lower(),i.lower()))\r
+               return identifiers\r
 \r
        RE_IDENTIFIER = re.compile(r"^ts(?:\((.+?)\))?:(.*)$")\r
 \r
@@ -244,8 +275,8 @@ class TouchHandler(threading.Thread):
                self._curTouchMode=mode\r
 \r
        def pump(self):\r
-               for tracker in self.trackerManager.emitTrackers():\r
-                       gesture=TouchInputGesture(tracker,self._curTouchMode)\r
+               for preheldTracker,tracker in self.trackerManager.emitTrackers():\r
+                       gesture=TouchInputGesture(preheldTracker,tracker,self._curTouchMode)\r
                        try:\r
                                inputCore.manager.executeGesture(gesture)\r
                        except inputCore.NoInputGestureAction:\r
index 3d8ccc0..bd92d62 100644 (file)
@@ -4,8 +4,10 @@
 #See the file COPYING for more details.\r
 #Copyright (C) 2012 NV Access Limited\r
 \r
+import threading\r
 import time\r
 from collections import OrderedDict\r
+from logHandler import log\r
 \r
 #Possible actions (single trackers)\r
 action_tap="tap"\r
@@ -54,16 +56,41 @@ class SingleTouchTracker(object):
        """\r
        Represents the lifetime of one single finger while its in contact with the touch device, tracking start and current coordinates, start and end times, and whether its complete (broken contact yet).\r
        It also calculates what kind of single action (tap, flick, hover) this finger is performing, once it has enough data.\r
+       @ivar ID: the ID this finger has been assigned by the Operating System.\r
+       @type ID: int\r
+       @ivar x: The last known x screen coordinate of this finger\r
+       @type x: int\r
+       @ivar y: The last known y screen coordinate of this finger\r
+       @type y: int\r
+       @ivar startX: The x screen coordinate  where the finger first made contact\r
+       @type startX: int\r
+       @ivar startY: The y screen coordinate  where the finger first made contact\r
+       @type startY: int\r
+       @ivar startTime: the time at which the finger first made contact\r
+       @type startTime: float\r
+       @ivar endTime: the time at which the finger broke contact. Before breaking contact the value is -1\r
+       @type endTime: float\r
+       @ivar maxAbsDeltaX: the maximum distance this finger has traveled on the x access while making contact\r
+       @type maxAbsDeltaX: int\r
+       @ivar maxAbsDeltaY: the maximum distance this finger has traveled on the y access while making contact\r
+       @type maxAbsDeltaY: int\r
+       @ivar action: the action this finger has performed (one of the action_* constants,E.g. tap, flickRight, hover etc). If not enough data has been collected yet the action will be unknown. \r
+       @type action: string\r
+       @ivar complete: If true then this finger has broken contact\r
+       @type complete: bool\r
        """\r
+       __slots__=['ID','x','y','startX','startY','startTime','endTime','maxAbsDeltaX','maxAbsDeltaY','action','complete']\r
 \r
        def __init__(self,ID,x,y):\r
                self.ID=ID\r
                self.x=self.startX=x\r
                self.y=self.startY=y\r
                self.startTime=time.time()\r
+               self.endTime=-1\r
                self.maxAbsDeltaX=0\r
                self.maxAbsDeltaY=0\r
                self.action=action_unknown\r
+               self.complete=False\r
 \r
        def update(self,x,y,complete=False):\r
                """Called to alert this single tracker that the finger has moved or broken contact."""\r
@@ -75,7 +102,8 @@ class SingleTouchTracker(object):
                absDeltaY=abs(deltaY)\r
                self.maxAbsDeltaX=max(absDeltaX,self.maxAbsDeltaX)\r
                self.maxAbsDeltaY=max(absDeltaY,self.maxAbsDeltaY)\r
-               deltaTime=time.time()-self.startTime\r
+               curTime=time.time()\r
+               deltaTime=curTime-self.startTime\r
                if deltaTime<multitouchTimeout: #not timed out yet\r
                        if complete and self.maxAbsDeltaX<maxAccidentalDrift and self.maxAbsDeltaY<maxAccidentalDrift:\r
                                #The completed quick touch never drifted too far from its initial contact point therefore its a tap\r
@@ -94,11 +122,35 @@ class SingleTouchTracker(object):
                                        self.action=action_flickUp\r
                else: #timeout exceeded, must be a kind of hover\r
                        self.action=action_hover\r
+               self.complete=complete\r
+               if complete:\r
+                       self.endTime=curTime\r
 \r
 class MultiTouchTracker(object):\r
-       """Represents an action joinly performed by 1 or more fingers."""\r
+       """Represents an action jointly performed by 1 or more fingers.\r
+       @ivar action: the action this finger has performed (one of the action_* constants,E.g. tap, flickRight, hover etc).\r
+       @type action: string\r
+       @ivar x: the x screen coordinate where the action was performed. For multi-finger actions it is the average position of each of the fingers. For plural actions it is based on the first occurence\r
+       @type x: int\r
+       @ivar y: the y screen coordinate where the action was performed. For multi-finger actions it is the average position of each of the fingers. For plural actions it is based on the first occurence\r
+       @type y: int\r
+       @ivar startTime: the time the action began\r
+       @type startTime: float\r
+       @ivar endTime: the time the action was complete \r
+       @type endTime: float\r
+       @ivar numFingers: the number of fingers that performed this action\r
+       @type numFingers: int\r
+       @ivar actionCount: the number of times this action was performed in quick succession (E.g. 2 for a double tap)\r
+       @ivar childTrackers: a list of L{MultiTouchTracker} objects which represent the direct sub-actions of this action. E.g. a 2-finger tripple tap's childTrackers will contain 3 2-finger taps. Each of the 2-finger taps' childTrackers will contain 2 taps.\r
+       @type childTrackers: list of L{MultiTouchTracker} objeccts\r
+       @ivar rawSingleTouchTracker: if this tracker represents a 1-fingered non-plural action then this will be the L{SingleTouchTracker} object for that 1 finger. If not then it is None.\r
+       @type rawSingleTouchTracker: L{SingleTouchTracker}\r
+       @ivar pluralTimeout: the time at which this tracker could no longer possibly be merged with another to be pluralized, thus it is aloud to be emitted\r
+       @type pluralTimeout: float\r
+       """\r
+       __slots__=['action','x','y','startTime','endTime','numFingers','actionCount','childTrackers','rawSingleTouchTracker','pluralTimeout']\r
 \r
-       def __init__(self,action,x,y,startTime,endTime,numFingers,actionCount,numHeldFingers):\r
+       def __init__(self,action,x,y,startTime,endTime,numFingers=1,actionCount=1,rawSingleTouchTracker=None,pluralTimeout=None):\r
                self.action=action\r
                self.x=x\r
                self.y=y\r
@@ -106,10 +158,30 @@ class MultiTouchTracker(object):
                self.endTime=endTime\r
                self.numFingers=numFingers\r
                self.actionCount=actionCount\r
-               self.numHeldFingers=numHeldFingers\r
+               self.childTrackers=[]\r
+               self.rawSingleTouchTracker=rawSingleTouchTracker\r
+               # We only allow pluralizing of taps, no other action.\r
+               if pluralTimeout is None and action==action_tap:\r
+                       pluralTimeout=startTime+multitouchTimeout\r
+               self.pluralTimeout=pluralTimeout\r
+\r
+       def iterAllRawSingleTouchTrackers(self):\r
+               if self.rawSingleTouchTracker: yield self.rawSingleTouchTracker\r
+               for child in self.childTrackers:\r
+                       for i in child.iterAllRawSingleTouchTrackers():\r
+                               yield i\r
 \r
        def __repr__(self):\r
-               return "<MultiTouchTracker {numFingers}finger {action} {actionCount} times at position {x},{y} with {numHeldFingers} extra fingers held>".format(action=self.action,x=self.x,y=self.y,numFingers=self.numFingers,actionCount=self.actionCount,numHeldFingers=self.numHeldFingers)\r
+               return "<MultiTouchTracker {numFingers}finger {action} {actionCount} times at position {x},{y}>".format(action=self.action,x=self.x,y=self.y,numFingers=self.numFingers,actionCount=self.actionCount)\r
+\r
+       def getDevInfoString(self):\r
+               msg="%s\n"%self\r
+               if self.childTrackers:\r
+                       msg+="--- made of ---\n"\r
+                       for t in self.childTrackers:\r
+                               msg+=t.getDevInfoString()\r
+                       msg+="--- end ---\n"\r
+               return msg\r
 \r
 class TrackerManager(object):\r
        """\r
@@ -119,83 +191,107 @@ class TrackerManager(object):
        def __init__(self):\r
                self.singleTouchTrackersByID=OrderedDict()\r
                self.multiTouchTrackers=[]\r
-               self.numHeldFingers=0\r
+               self.curHoverStack=[]\r
+               self.numUnknownTrackers=0\r
+               self._lock=threading.Lock()\r
+\r
+       def makePreheldTrackerFromSingleTouchTrackers(self,trackers):\r
+               childTrackers=[MultiTouchTracker(action_hold,tracker.x,tracker.y,tracker.startTime,time.time()) for tracker in trackers if tracker.action==action_hover]\r
+               numFingers=len(childTrackers)\r
+               if numFingers==0: return\r
+               if numFingers==1: return childTrackers[0]\r
+               avgX=sum(t.x for t in childTrackers)/numFingers\r
+               avgY=sum(t.y for t in childTrackers)/numFingers\r
+               tracker=MultiTouchTracker(action_hold,avgX,avgY,childTrackers[0].startTime,time.time(),numFingers)\r
+               tracker.childTrackers=childTrackers\r
+               return tracker\r
+\r
+       def makePreheldTrackerForTracker(self,tracker):\r
+               curHoverSet={x for x in self.singleTouchTrackersByID.itervalues() if x.action==action_hover}\r
+               excludeHoverSet={x for x in tracker.iterAllRawSingleTouchTrackers() if x.action==action_hover}\r
+               return self.makePreheldTrackerFromSingleTouchTrackers(curHoverSet-excludeHoverSet)\r
 \r
        def update(self,ID,x,y,complete=False):\r
                """\r
                Called to Alert the multiTouch tracker of a new, moved or completed contact (finger touch).\r
                It creates new single trackers or updates existing ones, and queues/processes multi trackers for later emition.\r
                """ \r
-               #See if we know about this finger\r
-               tracker=self.singleTouchTrackersByID.get(ID)\r
-               if not tracker:\r
-                       if not complete:\r
-                               #This is a new contact (finger) so start tracking it\r
-                               self.singleTouchTrackersByID[ID]=SingleTouchTracker(ID,x,y)\r
+               with self._lock:\r
+                       #See if we know about this finger\r
+                       tracker=self.singleTouchTrackersByID.get(ID)\r
+                       if not tracker:\r
+                               if not complete:\r
+                                       #This is a new contact (finger) so start tracking it\r
+                                       self.singleTouchTrackersByID[ID]=SingleTouchTracker(ID,x,y)\r
+                                       self.numUnknownTrackers+=1\r
+                               return\r
+                       #We already know about this finger\r
+                       #Update its position and completion status\r
+                       #But also find out its action before and after the update to decide what to do with it\r
+                       oldAction=tracker.action\r
+                       tracker.update(x,y,complete)\r
+                       newAction=tracker.action\r
+                       if (oldAction==action_unknown and newAction!=action_unknown):\r
+                               self.numUnknownTrackers-=1\r
+                       if complete: #This finger has broken contact\r
+                               #Forget about this finger\r
+                               del self.singleTouchTrackersByID[ID]\r
+                               if tracker.action==action_unknown:\r
+                                       self.numUnknownTrackers-=1\r
+                       #if the action changed and its not unknown, then we will be queuing it\r
+                       if newAction!=oldAction and newAction!=action_unknown:\r
+                               if newAction==action_hover:\r
+                                       #New hovers must be queued as holds \r
+                                       newAction=action_hold\r
+                               #for most  gestures the start coordinates are what we want to emit with trackers \r
+                               #But hovers should always use their current coordinates\r
+                               x,y=(tracker.x,tracker.y) if newAction in hoverActions else (tracker.startX,tracker.startY)\r
+                               #Queue the tracker for processing or emition\r
+                               self.processAndQueueMultiTouchTracker(MultiTouchTracker(newAction,x,y,tracker.startTime,time.time(),rawSingleTouchTracker=tracker))\r
+\r
+       def makeMergedTrackerIfPossible(self,oldTracker,newTracker):\r
+               if newTracker.action==oldTracker.action and  newTracker.startTime<oldTracker.endTime and oldTracker.startTime<newTracker.endTime and oldTracker.actionCount==newTracker.actionCount==1:\r
+                       #The old and new tracker are the same kind of action, they are not themselves plural actions, and their start and end times overlap\r
+                       #Therefore they should be treeted as one multiFingered action\r
+                       childTrackers=[]\r
+                       childTrackers.extend(oldTracker.childTrackers) if oldTracker.numFingers>1 else childTrackers.append(oldTracker)\r
+                       childTrackers.extend(newTracker.childTrackers) if newTracker.numFingers>1 else childTrackers.append(newTracker)\r
+                       numFingers=oldTracker.numFingers+newTracker.numFingers\r
+                       avgX=sum(t.x for t in childTrackers)/numFingers\r
+                       avgY=sum(t.y for t in childTrackers)/numFingers\r
+                       mergedTracker=MultiTouchTracker(newTracker.action,avgX,avgY,oldTracker.startTime,newTracker.endTime,numFingers,newTracker.actionCount,pluralTimeout=newTracker.pluralTimeout)\r
+                       mergedTracker.childTrackers=childTrackers\r
+               elif self.numUnknownTrackers==0 and newTracker.pluralTimeout is not None and newTracker.startTime>=oldTracker.endTime and newTracker.startTime<oldTracker.pluralTimeout and newTracker.action==oldTracker.action and oldTracker.numFingers==newTracker.numFingers:\r
+                       #The new and old action are   the same and allow pluralising and have the same number of fingers and there are no other unknown trackers left and they do not overlap in time\r
+                       #Therefore they should be treeted as 1 plural action (e.g. double tap)\r
+                       mergedTracker=MultiTouchTracker(newTracker.action,oldTracker.x,oldTracker.y,oldTracker.startTime,newTracker.endTime,newTracker.numFingers,oldTracker.actionCount+newTracker.actionCount,pluralTimeout=newTracker.pluralTimeout)\r
+                       mergedTracker.childTrackers.extend(oldTracker.childTrackers) if oldTracker.actionCount>1 else mergedTracker.childTrackers.append(oldTracker)\r
+                       mergedTracker.childTrackers.extend(newTracker.childTrackers) if newTracker.actionCount>1 else mergedTracker.childTrackers.append(newTracker)\r
+               elif self.numUnknownTrackers==0 and newTracker.action==action_hold and oldTracker.action==action_tap and newTracker.numFingers==oldTracker.numFingers and newTracker.startTime>oldTracker.endTime:\r
+                       #A tap and then a hover down  is a tapAndHold\r
+                       mergedTracker=MultiTouchTracker(action_tapAndHold,oldTracker.x,oldTracker.y,oldTracker.startTime,newTracker.endTime,newTracker.numFingers,oldTracker.actionCount,pluralTimeout=newTracker.pluralTimeout)\r
+                       mergedTracker.childTrackers.append(oldTracker)\r
+                       mergedTracker.childTrackers.append(newTracker)\r
+               else: #They don't match, go to the next one\r
                        return\r
-               #We already know about this finger\r
-               #Update its position and completion status\r
-               #But also find out its action before and after the update to decide what to do with it\r
-               oldAction=tracker.action\r
-               tracker.update(x,y,complete)\r
-               newAction=tracker.action\r
-               if complete: #This finger has broken contact\r
-                       #Forget about this finger\r
-                       del self.singleTouchTrackersByID[ID]\r
-                       #A completed hover should be a hoverUp\r
-                       if newAction==action_hover:\r
-                               newAction=action_hoverUp\r
-               #if the action changed and its not unknown, then we will be queuing it\r
-               if newAction!=oldAction and newAction!=action_unknown:\r
-                       if newAction==action_hover:\r
-                               #New hovers must be queued as hover down\r
-                               newAction=action_hoverDown\r
-                       #for most  gestures the start coordinates are what we want to emit with trackers \r
-                       #But hovers should always use their current coordinates\r
-                       x,y=(tracker.x,tracker.y) if newAction in hoverActions else (tracker.startX,tracker.startY)\r
-                       #We keep a count of held fingers as a modification  for gesutres.\r
-                       #We must decrement the count before queuing a hover up, but incrementing for hoverDown happens after queuing.\r
-                       #Otherwize hoverDowns and hoverUps would accidently get their own heldFinger modifiers.\r
-                       if oldAction==action_hover and newAction==action_hoverUp:\r
-                               self.numHeldFingers-=1\r
-                       #Queue the tracker for processing or emition\r
-                       self.addMultiTouchTracker(MultiTouchTracker(newAction,x,y,tracker.startTime,time.time(),1,1,self.numHeldFingers))\r
-                       if newAction==action_hoverDown:\r
-                               self.numHeldFingers+=1\r
-\r
-       def addMultiTouchTracker(self,tracker):\r
+               return mergedTracker\r
+\r
+       def processAndQueueMultiTouchTracker(self,tracker):\r
                """Queues the given tracker, replacing old trackers with a multiFingered plural action where possible"""\r
                #Reverse iterate through the existing queued trackers comparing the given tracker to each of them\r
                #as L{emitTrackers} constantly dequeues, the queue only contains trackers newer than multiTouchTimeout, though may contain more if there are still unknown singleTouchTrackers around.\r
                for index in xrange(len(self.multiTouchTrackers)):\r
-                       index=-1-index\r
+                       index=len(self.multiTouchTrackers)-1-index\r
                        delayedTracker=self.multiTouchTrackers[index]\r
-                       #We never want to merge actions if the held fingers modifier has changed at all\\r
-                       if tracker.numHeldFingers==delayedTracker.numHeldFingers:\r
-                               if tracker.action==delayedTracker.action and delayedTracker.startTime<=tracker.startTime<delayedTracker.endTime and delayedTracker.actionCount==tracker.actionCount==1:\r
-                                       #The old and new tracker are the same kind of action, they are not themselves plural actions, and their start and end times overlap\r
-                                       #Therefore they should be treeted as one multiFingered action\r
-                                       delayedTracker.numFingers+=tracker.numFingers\r
-                               elif tracker.action==action_tap==delayedTracker.action and delayedTracker.numFingers==tracker.numFingers:\r
-                                       #The new and old action are  both tap and have the same number of fingers, but they do not overlap in time\r
-                                       #Therefore they should be treeted as 1 plural action (e.g. double tap)\r
-                                       delayedTracker.actionCount+=tracker.actionCount\r
-                               elif tracker.action==action_hoverDown and delayedTracker.action==action_tap and tracker.numFingers==delayedTracker.numFingers:\r
-                                       #A tap and then a hover down is a tapAndHold\r
-                                       delayedTracker.action=action_tapAndHold\r
-                               else: #They don't match, go to the next one\r
-                                       continue\r
-                               #The old tracker's finger count or repete count was affected by the new tracker, therefore\r
-                               #Update the old tracker's times to match the new tracker, and remove an requeue the old action for further processing\r
-                               #Its necessary to reprocess to catch certain plural trackers (e.g. 1 finger tap turns to 2 finger tap, but later can match a previous 2 finger tap which makes a 2 finger double tap).\r
+                       mergedTracker=self.makeMergedTrackerIfPossible(delayedTracker,tracker)\r
+                       if mergedTracker:\r
+                               # The trackers were successfully merged\r
+                               # remove the old one from the queue, and queue the merged one for possible further matching\r
                                del self.multiTouchTrackers[index]\r
-                               delayedTracker.startTime=tracker.startTime\r
-                               delayedTracker.endTime=tracker.endTime\r
-                               self.addMultiTouchTracker(delayedTracker)\r
-                               break\r
-               else: #The new tracker did not affect any old tracker, so really queue it. \r
-                       if tracker.action!=action_hoverDown:\r
-                               self.multiTouchTrackers.append(tracker)\r
+                               self.processAndQueueMultiTouchTracker(mergedTracker)\r
+                               return\r
+               else:\r
+                       self.multiTouchTrackers.append(tracker)\r
 \r
        pendingEmitInterval=None #: If set: how long to wait before calling emitTrackers again as trackers are still in the queue \r
        def emitTrackers(self):\r
@@ -204,28 +300,48 @@ class TrackerManager(object):
                A part from a timeout, trackers are also not emitted if there are other fingers touching that still have an unknown action. \r
                If there are no queued trackers to yield but there is a hover tracker, a hover action is yielded instead.\r
                """\r
-               self.pendingEmitInterval=None\r
-               t=time.time()\r
-               hasUnknownTrackers=False\r
-               lastHoverTracker=None\r
-               #Check to see if there are any unknown trackers, and also find the most recent hover tracker if any.\r
-               for tracker in self.singleTouchTrackersByID.itervalues():\r
-                       if tracker.action==action_hover:\r
-                               lastHoverTracker=tracker\r
-                       if tracker.action==action_unknown:\r
-                               hasUnknownTrackers=True\r
-               foundTrackers=False\r
-               #Only emit trackers if there are not unknown actions\r
-               if not hasUnknownTrackers:\r
-                       for tracker in list(self.multiTouchTrackers):\r
-                               #All trackers can be emitted with no delay except for tap which must wait for the timeout (to detect plural taps)\r
-                               trackerTimeout=(tracker.startTime+multitouchTimeout)-t if tracker.action==action_tap else 0 \r
-                               if trackerTimeout<=0: \r
-                                       self.multiTouchTrackers.remove(tracker)\r
-                                       foundTrackers=True\r
-                                       yield tracker\r
-                               else:\r
-                                       self.pendingEmitInterval=min(self.pendingEmitInterval,trackerTimeout) if self.pendingEmitInterval else trackerTimeout\r
-               #If no tracker could be emitted, at least emit the most recent  hover tracker if there is one\r
-               if not foundTrackers and lastHoverTracker:\r
-                       yield MultiTouchTracker(lastHoverTracker.action,lastHoverTracker.x,lastHoverTracker.y,lastHoverTracker.startTime,t,1,1,self.numHeldFingers-1)\r
+               with self._lock:\r
+                       self.pendingEmitInterval=None\r
+                       t=time.time()\r
+                       # yield hover ups for complete hovers from most recent backwards\r
+                       for singleTouchTracker in list(self.curHoverStack): \r
+                               if singleTouchTracker.complete:\r
+                                       self.curHoverStack.remove(singleTouchTracker)\r
+                                       tracker=MultiTouchTracker(action_hoverUp,singleTouchTracker.x,singleTouchTracker.y,singleTouchTracker.startTime,time.time())\r
+                                       preheldTracker=self.makePreheldTrackerFromSingleTouchTrackers(self.curHoverStack)\r
+                                       yield preheldTracker,tracker\r
+                       #Only emit trackers if there are not unknown actions\r
+                       hasUnknownTrackers=self.numUnknownTrackers\r
+                       if not hasUnknownTrackers:\r
+                               for tracker in list(self.multiTouchTrackers):\r
+                                       # isolated holds can be dropped as we only care when they are tapAndHolds (and preheld is handled later)\r
+                                       #All trackers can be emitted with no delay except for tap which must wait for the timeout (to detect plural taps)\r
+                                       trackerTimeout=tracker.pluralTimeout-t if tracker.pluralTimeout is not None else 0 \r
+                                       if trackerTimeout<=0: \r
+                                               self.multiTouchTrackers.remove(tracker)\r
+                                               # isolated holds should not be emitted as they are covered by hover downs later\r
+                                               if tracker.action==action_hold:\r
+                                                       continue\r
+                                               preheldTracker=self.makePreheldTrackerFromSingleTouchTrackers(self.curHoverStack)\r
+                                               # If this tracker was made up of any new hovers (e.g. a tapAndHold) they should be quietly added to the current hover stack so that hover downs are not produced\r
+                                               for singleTouchTracker in tracker.iterAllRawSingleTouchTrackers():\r
+                                                       if singleTouchTracker.action==action_hover and singleTouchTracker not in self.curHoverStack:\r
+                                                               self.curHoverStack.append(singleTouchTracker)\r
+                                               yield preheldTracker,tracker\r
+                                       else:\r
+                                               self.pendingEmitInterval=min(self.pendingEmitInterval,trackerTimeout) if self.pendingEmitInterval else trackerTimeout\r
+                       # yield hover downs for any new hovers\r
+                       # But only once  there are no more trackers in the queue waiting to timeout (E.g. a hold for a tapAndHold)\r
+                       if len(self.multiTouchTrackers)==0:\r
+                               for singleTouchTracker in self.singleTouchTrackersByID.itervalues():\r
+                                       if singleTouchTracker.action==action_hover and singleTouchTracker not in self.curHoverStack:\r
+                                               self.curHoverStack.append(singleTouchTracker)\r
+                                               tracker=MultiTouchTracker(action_hoverDown,singleTouchTracker.x,singleTouchTracker.y,singleTouchTracker.startTime,time.time())\r
+                                               preheldTracker=self.makePreheldTrackerFromSingleTouchTrackers(self.curHoverStack[:-1])\r
+                                               yield preheldTracker,tracker\r
+                       # yield a hover for the most recent hover\r
+                       if len(self.curHoverStack)>0:\r
+                               singleTouchTracker=self.curHoverStack[-1]\r
+                               tracker=MultiTouchTracker(action_hover,singleTouchTracker.x,singleTouchTracker.y,singleTouchTracker.startTime,time.time())\r
+                               preheldTracker=self.makePreheldTrackerFromSingleTouchTrackers(self.curHoverStack[:-1])\r
+                               yield preheldTracker,tracker\r
index 24357cf..5c0b1b4 100644 (file)
@@ -41,6 +41,8 @@
 - When speaking line indentation, non-breaking spaces are now treated as normal spaces. Previously, this could cause announcements such as "space space space" instead  of "3 space". (#5610)\r
 - When closing a modern Microsoft input method candidate list, focus is correctly restored to either the input composition or the underlying document. (#4145)\r
 - In Microsoft Office 2013 and later, when the ribbon is set to show only tabs, items in the ribbon are again reported as expected when a tab is activated. (#5504)\r
+- Fixes and improvements to touch screen gesture detection and binding. (#5652)\r
+- Touch screen hovers are no longer reported in input help. (#5652)\r
 \r
 \r
 == Changes for Developers ==\r
  - hwIo.Serial extends pyserial to call a callable when data is received instead of drivers having to poll.\r
  - hwIo.Hid provides support for braille displays communicating via USB HID.\r
  - hwPortUtils and hwIo can optionally provide detailed debug logging, including devices found and all data sent and received.\r
+- There are several new properties accessible from touch screen gestures: (#5652)\r
+ - MultitouchTracker objects now contain a childTrackers property which contains the MultiTouchTrackers the tracker was composed of. For example, 2 finger double tap has child trackers for two 2-finger taps. The 2-finger taps themselves have child trackers for two taps.\r
+ - MultiTouchTracker objects now also contain a rawSingleTouchTracker property if the tracker was the result of one single finger doing a tap, flick or hover. The SingleTouchTracker allows access to the underlying ID assigned to the finger by the operating system and whether or not the finger is still in contact at the current time.\r
+ - TouchInputGestures now have x and y properties, removing the need to access the tracker for trivial cases.\r
+ - TouchInputGesturs now contain a preheldTracker property, which is a MultitouchTracker object representing the other fingers held while this action was being performed.\r
+- Two new touch screen gestures can be emitted: (#5652)\r
+ - Plural tap and holds (e.g. double tap and hold)\r
+ - A generalized identifier with finger count removed for holds (e.g. hold+hover for 1finger_hold+hover).\r
 \r
 \r
 = 2015.4 =\r