Input on current AAC Devices

[lgelb]

There are some free iOS TTS apps that I haven't tried, but the iOS itself enables any highlighted text to be spoken once "Speak selection" is turned on in the accessibility settings. So anyone who could still type could use the virtual kb to type the text s/he wanted spoken. You can also turn on speech as each key is typed.

DK anything re Android in that respect, but would think it would have similar capabilities --

 

[notBrad]

Thanks. I was thinking of some along the lines of using word clouds as a way of making it easier for users to put together phrases, etc... and construct sentences without having to type anything.

I've noticed in programming health care software for progress notes that people tend to have certain identifiable patterns in the way they talk so I got to thinking that something that would present their most commonly used phrases "front and center" so to speak might make it easier for the user to quickly jump around between their most used phrasing and rapidly assemble sentences without the need to type things. Hopefully the end effect would be to make them more "conversational" with less effort.

If it was browser based then someone could use it from a variety of devices without needing to have an expensive dedicated device. Heck, if the personalized parameters (commonly used phrases, alternate phrasing, etc...) where retrievable from a cloud based storage the user could use any variety of devices depending on what they had available at the moment (Oh, left the tablet at home but the smart phone will work in a pinch).

Just kinda thinking out loud from reading the various comments...

 

[dwpaul]

Hi everyone,

I’m really interested in this thread on ACC and was wondering if I could get your feedback on a new ACC device my team is working on.

XAVIER is thought-to-text software in development by students and alumni of Johns Hopkins University and the University of Southern California, aimed at allowing users to create text on their computerized device through thought. Basically, users wearing a brain-computer-interface headset can simply visualize words appearing on their computerized device; our software processes their neural signals in order to predict and produce the users intended word as text or synthesized speech. This software was conceived as a new Speech Generating Device (SGD) for use by individuals with severe speech impairment since current SGDs have several limitations including prohibitive cost and difficulty of use. The conventional speech generating devices have a significantly slower rate of voice output than natural speech and also require some muscle movement for their operation. As muscular degeneration progresses, they become increasingly difficult to operate.

The idea is that our software removes motor function from the equation, breaking down the last barrier between the individual and their computer. We hope that Xavier will be less expensive, faster, and easier to use than any currently available speech generating device or speech to text software.

Do you think individuals in the community would be interested in this software? Please let me know if you have any questions or suggestions for us!

Thanks so much for your time and consideration!

 

[lgelb]

Sorry, NB, I should've pointed you to the Dasher site, then. Browser-based, word-cloud kinda thing for speedier typing. Way too dizzying for my PALS but others might find it of benefit, and I'm sure there are ways to improve it.

 

[notBrad]

Thanks for the heads up and no worries. I'll check it out and see what's up. I'm one of those creative types who's always looking at stuff and going "you know what that could use to function better...". If it's something I could come up with a better solution for then I might see what I can do. Worst case it'd be a good programming exercise and I could put it out there for free and maybe someone could use it or take it and run with it.

 

[tawill]

Natural speech, as in a donor voice? My speech is very difficult to understand by almost anyone, so I can't imagine matching up a word from my voice. I am anxious to check Alpha Topics out though. I have been using my Xcel with speech, and have had excellent luck. It does require more typing. The great thing is you only have to type and enter for it to speak. Of course your arrow keys will repeat any phrase or sentence you want to. Word was a few more steps, by typing, highlighting, then icon click. Still works though. I would love to know more about the thought to speech or text. Right now I don't mind typing, but I have noticed an increase in cramping in both my hands the last few days .

 

[notBrad]

Sorry, NB, I should've pointed you to the Dasher site, then. Browser-based, word-cloud kinda thing for speedier typing. Way too dizzying for my PALS but others might find it of benefit, and I'm sure there are ways to improve it.

Thanks for the link to that. Not to put down their efforts but I know I can build a MUCH better/easier to use user interface.

I'm going to mock something up and maybe provide a link to get some feedback, maybe a kickstarter to help offset the cost some...

 

[Nikki J]

Anything you link to a fundraiser such as kickstarter is not allowed unfortunately.

 

[notBrad]

Oh no no no, I would never link anything like in a forum.

I was just mentioning that I was thinking of mocking something up and if it got enough positive response and seemed like it might be a usable solution then I would probably go out to a crowd source site and try to get enough funding to be able to finish it. That was all. Sorry it if came across as otherwise.

 

[Nikki J]

Thanks. Sorry I misunderstood. I look forward to seeing what you come up with

 

[notBrad]

No apologies needed, I went back and read what I wrote and I definitely should have been clearer. Any misunderstanding was totally my fault.

Will try and get something up next week.

 

[lgelb]

Brad,
If you are looking to better Dasher (free), you might also want to check out the time-limited demos of the Mac virtual keyboards/speech generators at assistiveware (the full versions cost, of course, which is why your notion would be generous) just to see how they handle word prediction. Likewise the paid apps that some people here use on iOS, which you can find in other threads or people can post here.

 

[notBrad]

Thanks and I will check those out.

There are sooo many free TTS applications and APIs out there that it just doesn't take that much to string them together and create something that folks could run from a web browser thereby making it pretty universal.

There's a technology called Tag Clouding which is a way of viewing web page content by frequency/importance and my thought is that combined with some crypto algorithms (they depend on word and letter frequency as a starting point for many operations) should be able to create a fairly easy to use speech generation system.

So my thoughts were that I could mock up some UI stuff and get feedback from folks who would actually use it, the create a proof of concept to see if it was actually functional enough to be of use for folks and if so go to a crowdfunding site like kick starter and see if I could get enough funding to be able to actually create a production version.

I'd think that making it freeware would help with that and also just as importantly by making it open source and freeware I'd open the doors for contributions (programming wise not $$$) to anyone who had an idea to make it better and equally important put it in the public domain which would prevent someone from trying to patent it and pushing the free version out of the market.

See once it's out there published and free it belongs to the public and any attempts to "privatize" it wouldn't work.

I design and program complex distributed systems BTW, specifically what they call "data-driven" systems which IMO lends itself to this particular problem.

 

[notBrad]

OK, here is a VERY, VERY ROUGH mockup that is really more of just a proof of concept That I threw together in a few hours

TagSpeak - Tag based TTS program

You can click on one of the buttons at the top and it will present a word cloud (or listing in the case of individual letters) that can be clicked on to create text at the bottom.

As you mouse over the word clouds they rotate and move which *should* make it easier to get to a word with less movement. Clicking on a word or phrase in the cloud concatenates it with the others at the bottom of the page which you can select and have a TTS engine speak (I tested with FoxVox in FireFox). Obviously in a production version you'd simply click a link telling the TTS app to speak it. I've even thought of having a setting that would kick it in after a comma or some other "clue" which would make it more conversational.

Several things to keep in mind

1. This is just a proof of concept not anything approaching a finished app. So bugs, poor performance and general goofiness are to be expected.

2. Everything can "set-able" on a per user basis (rotation, speed, type, size, etc...) so feedback/input on what directions would be of value would be greatly appreciated.

3. The words/phrases can be sized, high-lighted, etc... according to frequency of use

4. The words and letter (as well as diphthongs, etc...) can be in placed in libraries that would allow a user to create their own personalized settings.

5. My thinking is that eventually a user would have the ability to switch between customer phrase libraries as well as augment them with standard stuff (most common three letter words, etc...) that would allow them the ability to pick one's corresponding to a particular social setting or conversation need such work, restaurant, hanging out with friends and so on.

6. The system would track how often a user used certain words and place them into the cloud with a heavier ranking thereby eventually creating a library tailored to that user in terms of word prediction.

By making it browser based they could use it with any device that could use a browser and by allowing them to store their personal libraries locally they wouldn't be dependent on internet access all the time.

I'm thinking that by using crypto libraries (these are ones that list letters most likely to follow "E", or the most common two letter combinations and things like that) I could create clouds that users could jump to very quickly to create sentences. That coupled with the ability to save their own favorite phrases could make it a lot easier for them to quickly build a personalized library unique to their style and way of communicating.

Anyway, like I said, it's a programming exercise I whipped out pretty quickly and if there's enough good suggestions and feedback I'll keep at it and see what can be done to get it into the public domain for free.

 

[dwpaul]

Nice job with TagSpeak notbrad.