Journal of Current Scientific Research

Journal of Current Scientific Research

Current Issue Volume No: 2 Issue No: 1

Research-article Article Open Access
  • Available online freely Peer Reviewed
  • Effects Of Feedback Of Fingertip Force Information With Temporal Coded Vibration Stimulation On Precision Grasping Tasks

    1 Department of Information and Communications Engineering, Tokyo Institute of Technology. 

    2 Institute of Innovative Research, Tokyo Institute of Technology 

    Abstract

    Tactile information plays an important role in human manipulation of objects; however, prosthetic limb placement or teleoperation requires the manipulation of alternative bodies in the absence of tactile sensations. To provide an alternative to tactile sensation, this study proposes and assesses a continuous feedback scheme with temporally coded vibration. This scheme was designed to provide discrete intended tactile information in response to changing object-controlled situations by repetitively presenting time-coded vibration patterns. The effects of the proposed scheme on an object with acatch-and-hold task in virtual reality were confirmed. Compared to the control feedback scheme that provides vibration only when the balance of the virtual grip force and object position is changed, the proposed feedback scheme has a better effect in terms of the success rate of holding on to the object with an appropriate holding force during the task. The effect is larger, especially in the invisible task condition, suggesting that the increased amount of information with coded vibration patterns can be used without any special training, especially without visual information. Considering the existing studies that show the effect of a feedback scheme in response to motion events, the continuous feedback scheme proposed in this study may be more suitable for movements that require sequential coordination and passive responses than stimulation methods based on motion events. This feedback scheme has potential applications not only in tele-technology but also in healthcare, such as rehabilitation.

    Author Contributions
    Received Dec 14, 2022     Accepted Dec 24, 2022     Published Dec 31, 2022

    Copyright© 2022 Hayashi Tomoya, et al.
    License
    Creative Commons License   This work is licensed under a Creative Commons Attribution 4.0 International License. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

    Competing interests

    The authors have declared that no competing interests exist.

    Funding Interests:

    Citation:

    Hayashi Tomoya, Yoshimura Natsue (2022) Effects Of Feedback Of Fingertip Force Information With Temporal Coded Vibration Stimulation On Precision Grasping Tasks Journal of Current Scientific Research. - 2(1):5-17
    DOI 10.14302/issn.2766-8681.jcsr-22-4400

    Introduction

    Introduction

    Feedback information from motion is important for controlling and learning physical tasks. Without error information between the intended and actual motions, we cannot obtain a clue to correct the next motion 1, 2. Tactile information is known to act as feedback for adjusting motion control and learning behavior 3, 4, and 5. Recently, movable prosthetic limbs, virtual reality systems, and teleoperation of robots have required users to operate their alternative bodies without tactile feedback 6, 7, and 8. In another case, patients with sensory paralysis lacked tactile feedback from their limbs even though their motor functions were normal 9, 10. These cases force people to rely on small visual or auditory signals to understand the situation.

    Artificial tactile feedback is expected to play a complementary role to the information obtained from natural tactile sensations. Among the various possible artificial tactile feedbacks, vibration stimulation has the capabilities of compactness, ease of maintenance, and recognizability. Vibration stimulation comprises several variables, such as amplitude and frequency, and feedback schemes have been proposed that directly link these variables to target tactile information or the magnitude of the force 11, 12, and 13. However, because human tactile sensitivity is affected by vibration variables14, 15, and 16, as well as by the wearing condition of the vibration device and the adapting, the information feedback using the variables raises concerns about robustness.

    As a remedy for unreliability, several feedback methods have been proposed with conversion and discretization, such as changing the stimulation position 17, 18, and 19 and timing corresponding to motion events that include touch, lift-up, and break 20, 21. The methods that notify users only when a motion event occurs have been confirmed to be sufficiently effective for simple single-operation tasks 20, 21, and 22; however, it is questionable whether the amount of feedback information is sufficient in an environment that requires continuous passive responses to motion events, such as when the interaction with objects is dynamic.

    Therefore, in this study, we propose a continuous vibration coding scheme as a novel tactile feedback method that increases the amount of feedback information and assesses its effect with a catch-and-hold task that requires fine-tuning and holding for several seconds in a virtual environment. By setting up vibration temporal patterns that correspond discretely to the target tactile information, we expect that this scheme can ensure the reliability of information transfer through stimulation while preserving the amount of information.

    Results

    Results

    Figure 2 shows the results for Success Figure 2 (A), Fail Figure 2 (B), and Unsuccess Figure 2 (C) rates as performances. An ANOVA was conducted for each rate to ascertain which feedback contributed to task performance. The results Figure 2 (A) showed no significant interaction for the success rate (F = 1.34, p > .05), and the main effects were found for the vibration (F = 9.17, p <.001) and visibility (F = 124, p <.001) conditions. The post-hoc tests indicated that the success rate was significantly higher in the continuous-coded condition than in the other three conditions and that the success rate was significantly higher in the visible condition. However, a significant interaction was found for the FAIL rate (F = 3.61, p < .05) Figure 2 (B). The post-hoc test indicated significant differences between the visible and invisible conditions for each vibration condition (p < .01), but no significant differences were found between the vibration conditions for both visible and invisible conditions (p > .05), although continuous-coded conditions showed a relatively low failure rate. The results of the unsuccess rate showed no significant interaction (F = 0.45, p > .05) Figure 2(C), and the main effects were found for the vibration condition (F = 3.65, p < .05) but not for the visibility condition (F = 2.17, p > .05). The post-hoc test indicated that the unsuccess rate for continuous-coded was significantly lower than that for single-coded (p < .01).

    Next, we investigated the trajectories of the holding force to determine whether the continuous-coded condition had an effect on maintaining a stable holding during the trials. Figure 3 (A) shows examples of the trajectories of one participant for each object’s weight in the invisible condition. With the continuous-coded vibration (red lines), the holding force initially fluctuated and later remained within a “stable hold” range (blue ranges). The total time at “stable hold” for trials other than failed trials was extracted as an index of control precision and is compared in Figure 3 (B). The continuous-coded group showed the longest “stable hold” time, and ANOVA showed a significant interaction (F = 5.55, p < .01). The post-hoc test showed that the time for the continuous-coded condition was significantly longer than for the no condition, regardless of visibility conditions (p < .01). The time for the continuous-coded condition was also longer than for the single-coded in the visible condition (p < .01). In addition, the duration of the single-coded condition was significantly longer than that of the no condition in the invisible condition (p < .05). The time in the visible condition was significantly longer than that in the invisible condition for all vibration conditions (p < .01).

    To examine the reason why total “stable hold” time increased with the continue-coded, we additionally calculated the rate of returning to a “stable hold” state after transitions from “stable hold” to “overpowered” or “weak hold” (Figure 4). For the return rate from “overpowered” Figure 4(A), no significant interaction was indicated (F = 1.12, p > .05) and both main effects of vibration (F = 11.2, p < .001) and visibility (F = 10.2, p < .01) were confirmed. As a result of the post-hoc test, the continue-coded condition had a significantly high returning rate from “overpowered” than that in the none condition (p < .01), and the visible condition had a significantly higher rate than the invisible condition (p < .01). For the return rate from “weak hold” Figure 4 (B), a significant interaction was found (F = 8.06, p< .001). Although there was no difference in the visible condition, the rate from the “weak hold” of the none condition was significantly lower than that of the continuous-coded condition (p < .01) and the single condition (p < .001) in the invisible condition. For each vibration condition, the rate from the “weak hold” in the visible condition was significantly higher than in the invisible condition (p < .01).

    Comparisons of each performance rate. (A) Success rate. (B) Fail rate. (C) Unsuccess rate. The blue and orange bars indicate the performance of each vibration condition including both visible and invisible cases at success rate (A) and unsuccess rate (C), respectively, because Success and Unsuccess rates showed no significant interaction (F = 1.34, p > .05 for success rate, F = 0.45, p > .05 for unsuccess rate). At fail rate (B), filled and vacant bars represent the performance in visible and invisible conditions, respectively. Significant differences are marked only for those between the vibration conditions. Comparisons focusing on holding force trajectories and total "stable hold” time. (A) Examples of holding force trajectories. The left graph shows the trajectories with a light object, and the right graph shows those with a heavy object. The black solid, dotted, dashed, and red solid lines represent force transitions in the none, single, single-coded, and continuous-coded conditions, respectively. (B) Total time in “stable hold”. The filled and vacant bars represent the time in visible and invisible conditions, respectively. Significant differences are marked only for those between the vibration conditions. Return rates to “stable hold”. (A) Return rate from “overpowered”. The bars indicate the rate of each vibration condition, including the rate of both visible and invisible, as ANOVA did not show an interaction effect (F = 1.12, p > .05). (B) Return rate from “weak hold”. The filled and vacant bars represent the performance in visible and invisible conditions, respectively. Significant differences are marked only for those between the vibration conditions.

    Discussion

    Discussion

    In this study, we demonstrated that the proposed tactile feedback scheme based on continuous-coded vibration stimulation had an enhanced effect on catch-and-hold object task performance in a VR environment. The success rate of the task was significantly higher in the continuous-coded condition than in other conditions. However, the Fail (i.e., insufficient force) and Unsuccess (i.e., excessive force) rates did not show significant differences. The results indicate that the type of vibration stimulation affects the fine-tuning of holding force.

    Looking at the force shift during the trial in more detail as shown in Figure 3 (A), the continuous-coded condition showed superior performance for total stable hold time than that for the none and single-coded conditions. In addition, the force was more stable in the stable-hold range under the continuous-coded condition. Therefore, it is considered that the continuous-coded vibration method can assist in quick adjustment to the appropriate state and assist in fine-tuning even within the same state to avoid shifting the state. The return rates to the stable hold from other statuses (weak-hold and overpowered) also showed the superiority of the continuous-coded condition, supporting the concrete effect of the proposed method in fine-tuning holding even under invisible conditions. The fact that single- and single-coded conditions did not have as positive an effect as a continuous-coded condition suggests that repeated stimulation made participants less likely to miss the information presented by the vibration and more likely to keep track of their own holding state. In feedback control, it is important to ensure the robustness of the information to determine the direction of correction. On the other hand, considering the requirements of Success not being an overpowered state, the stimulation method does not only affect the immediate recovery of the state but also the avoidance of state transitions before they occurred. It is possible that the continuous-coded vibration affected th feedforward coordination in the prediction of the initial response and adjustment amount.

    The single condition that is based on the event-related method, for which performance improvements have been reported 20, 21, and 22, had only limited positive effects, with only a return rate from weak hold in the invisible condition. In our study, the task was longer and more complex than in previous studies. The differences in the tasks may have affected the attenuation of the support effect for controlling the holding force. Another study also reported a result in which the stimulation method based on the contact event had no significant effect in a task longer than five seconds, but the study reported no significant slip prevention effect when the method did not provide stimulation with an object slipped 28. The positive effect of a return rate from weak hold in an invisible condition with a single condition compared to a none condition Figure 4 (B) indicates that the single stimulation method allows for dealing with the particular situations to be avoided even if the task time is longer when slip events are also notified. On the other hand, the other performances, including the return rate from the overpowered state, did not show a pronounced effect of the single condition. The transition to weak hold and overpowered, in which the same stimulus was presented with a single condition, was difficult to distinguish in the invisible condition. As a result, it is considered that a movement toward recovery from weak hold, which leads to a critical situation of the object falling, was preferentially induced. This suggests that a simple feedback method, such as a single condition, may not be sufficiently effective for complex movements in which simple feedback and responses cannot be handled.

    The single-coded condition added the information by vibration patterns to the single condition but did not show improved assistive effects compared to the single condition, contrary to our expectation. However, the condition showed a significant effect on the total stable hold time Figure 3 (B), a more fundamental index, in an invisible situation compared to the no condition. The increase in the total time of the optimal state despite the lack of a clear effect on either immediate correction or advance prediction of control suggests that the single-coded method promotes fine-tuning to prevent state transitions within the same state. Because this role is also observed for the continuous-coded condition, it is considered to be an effect of the stimulation coding method. However, the effect of the single-coded condition is limited to an invisible situation, perhaps because the complexity of the stimulus and low presentation probability result in less reliability for the vibrating stimulus, and the effect Is lost in a visible environment where multiple information presentations are mixed.

    In summary, we found that the feedback method with continuous-coded vibration was effective for manipulation in tasks that were not completed by feedforward control. The method of coded vibration stimulation assisted in fine-tuning the manipulation by support in state awareness, and the addition of continuous stimulation to this ensured the robustness of feedback information. The results support both hypotheses of the assist effect of the increased information content of the stimulus and the certainty effect of the continuous stimulation; however, they also indicate that the increased information content does not function well on its own.

    Conclusion

    Conclusion

    This study examined whether the conversion of innate haptic information into an artificial stimulation of vibration, especially when coded, is adequate for motion control. The continuous-coded vibration feedback, that is, the proposed method, worked effectively for complex and time-consuming holding movements. This result is considered to be because of the assistance in understanding the manipulation state by stimulus coding and the robustness of the information provided by the continuous stimulus. However, the control conditions of the feedback method, which showed good effects in previous studies, had only limited effects in our task, suggesting that the appropriate type of feedback stimulus may differ depending on the task. We believe that the present encoded stimulus, which takes 300 ms for single feedback, should be evaluated in the future to determine if it is effective for short-term tasks, such as those in the previous studies. Exploring the characteristics of work tasks and suitable feedback stimulation methods will lead to tools that support rehabilitation and enrich the daily lives of prosthetic hand users and paraplegic patients.

    Affiliations:
    Affiliations: