Long story short, I have a friend that recently became paralysed from the
neck down, and is unable to speak because of a breathing tube. He’s very
much aware, but has no way to communicate, other than yes and no (and
sometimes that’s difficult to discern). Anywho, I am writing a
program/interface that’s main goal is to have eye tracking as a means
interaction, and most of my google searches lead me to OpenCV, and I’ve
found a few video demos that were apparently made with SDL and OpenCV. Does
anyone have some links or reference on how to combine these two libraries,
and further, to do eye/gaze tracking.
The concept of the UI is that when he looks at a part of the screen for more
than a secord or so, the UI assumes he has selected that button or option.
Of course, I’ll leave some room for some fine tuning, as my friend is fairly
medicated for the time being.
As far as equipment goes, I’m going to find the best webcam I can for him,
for the fact that the more accurate the images are, the more accurate I can
do eye tracking.
Up until now, I haven’t done a project like this, so any help anyone can
give would be wonderful.
Can he blink?
It would be faster than staring at the same spot for a second.
To avoid mistakes, you could have him blink twice rather than once,
and assume a single blink is involuntary.
I’m no expert, I’ve never tried something like this, but here are my
thoughts on the subject:
You’d probably have to configure it before each session since his
head, the screen and the webcam probably won’t all be in exactly the
same position each time. Something like: look at the center and
blink, look at each corner in turn and blink, look at the center
again and blink. He would probably have to keep his head very still
throughout the session for it to keep working, or your software would
have to track the movement of his head (not just position, but angle
as well).
Keep in mind, eye movements aren’t so dramatic as waving a hand. You
may not be able to track eye movements with a webcam. You need high
resolution, high framerate, high contrast, and you need to zoom in
close on his face.
You may also want to check out Tobii Technology ( http://www.tobii.com/corporate/start.aspx ) which is working on
similar applications to what you’re doing here.On 02/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
The concept of the UI is that when he looks at a part of the screen for more
than a secord or so, the UI assumes he has selected that button or option.
I remember watching the Diving Bell and the Butterfly (“Le Scaphandre
et le Papillon” if you speak French). They had an appalling system
where one person would say each letter of the alphabet in turn and the
other would blink to indicate which letter he wanted. I still think
binary or morse code would be a whole lot easier, even with memorizing
code tables…
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the other
covered from the camera, but pointed at his eye. I put the camera on
infrared mod, and I should get the light near the middle of his face, and
the light reflecting off his eye - from there, I’d assume the pupil would
reflect the most, so I just compare the distance/angle between the two
lights to approximate an x/y coordinate pair.
Does this sound reasonable? This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the last one
for a reference point.
I remember watching the Diving Bell and the Butterfly (“Le Scaphandre
et le Papillon” if you speak French). They had an appalling system
where one person would say each letter of the alphabet in turn and the
other would blink to indicate which letter he wanted. I still think
binary or morse code would be a whole lot easier, even with memorizing
code tables…
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the other
covered from the camera, but pointed at his eye. I put the camera on
infrared mod, and I should get the light near the middle of his face, and
the light reflecting off his eye - from there, I’d assume the pupil would
reflect the most, so I just compare the distance/angle between the two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. Light reflects off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the last one
for a reference point.
You should only need one light. Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. That’s before you actually get around to
figuring out where he’s looking though.On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done, i’ll post
the code to link it up with SDL’s events (probably as a user-event), as well
as a demo application.
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the other
covered from the camera, but pointed at his eye. I put the camera on
infrared mod, and I should get the light near the middle of his face, and
the light reflecting off his eye - from there, I’d assume the pupil would
reflect the most, so I just compare the distance/angle between the two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. Light reflects off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the last
one
for a reference point.
You should only need one light. Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. That’s before you actually get around to
figuring out where he’s looking though.
This sounds very cool, I’d love to hear what comes of this! :)On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done, i’ll post
the code to link it up with SDL’s events (probably as a user-event), as well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the
other
covered from the camera, but pointed at his eye. ?I put the camera on
infrared mod, and I should get the light near the middle of his face,
and
the light reflecting off his eye - from there, I’d assume the pupil
would
reflect the most, so I just compare the distance/angle between the two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. ?Light reflects off
both the outer and inner surface of the lens. ?If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. ?You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. ?You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. ?So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? ?This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the last
one
for a reference point.
You should only need one light. ?Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. ?You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. ?That’s before you actually get around to
figuring out where he’s looking though.
Right now I’m working on building the glasses/camera system, but I’m at my
parent’s place helping them out this week, so probably closer to the end of
the week, I’ll have something working, maybe with some configuration stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping EyeWriter
apart and just using opencv (they use openframeworks for their project, and
that adds a lot of bulk to a project that doesn’t need it).
Take care,
-AlexOn Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet is something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done, i’ll
post
the code to link it up with SDL’s events (probably as a user-event), as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the
other
covered from the camera, but pointed at his eye. I put the camera on
infrared mod, and I should get the light near the middle of his face,
and
the light reflecting off his eye - from there, I’d assume the pupil
would
reflect the most, so I just compare the distance/angle between the two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. Light reflects off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the
last
one
for a reference point.
You should only need one light. Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. That’s before you actually get around to
figuring out where he’s looking though.
My website just went down, so I remembered my google pages website, and
converted it to my new programming site.
Check out the projects page to keep up to date with my progress. I now have
the camera system built and have done some testing and I think within this
week, I’ll have an official release.
Take care,
-AlexOn Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <@Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system, but I’m at my
parent’s place helping them out this week, so probably closer to the end of
the week, I’ll have something working, maybe with some configuration stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for their
project, and that adds a lot of bulk to a project that doesn’t need it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet is something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done, i’ll
post
the code to link it up with SDL’s events (probably as a user-event), as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the
other
covered from the camera, but pointed at his eye. I put the camera on
infrared mod, and I should get the light near the middle of his face,
and
the light reflecting off his eye - from there, I’d assume the pupil
would
reflect the most, so I just compare the distance/angle between the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. Light reflects off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the
last
one
for a reference point.
You should only need one light. Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. That’s before you actually get around to
figuring out where he’s looking though.
Nice! :)On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <alex.barry at gmail.com> wrote:
My website just went down, so I remembered my google pages website, and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress.? I now have
the camera system built and have done some testing and I think within this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <alex.barry at gmail.com> wrote:
Right now I’m working on building the glasses/camera system, but I’m at my
parent’s place helping them out this week, so probably closer to the end of
the week, I’ll have something working, maybe with some configuration stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for their
project, and that adds a lot of bulk to a project that doesn’t need it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga <@slouken> wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done, i’ll
post
the code to link it up with SDL’s events (probably as a user-event), as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes, the
other
covered from the camera, but pointed at his eye. ?I put the camera
on
infrared mod, and I should get the light near the middle of his
face,
and
the light reflecting off his eye - from there, I’d assume the pupil
would
reflect the most, so I just compare the distance/angle between the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. ?Light reflects off
both the outer and inner surface of the lens. ?If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. ?You need a really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. ?You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. ?So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? ?This would also make it easy for blink
detection, and I could use three lights, one for each eye, and the
last
one
for a reference point.
You should only need one light. ?Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not just
wherever you point it. ?You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. ?That’s before you actually get around to
figuring out where he’s looking though.
The code is fairly messy and not well documented.
I am using my own custom UI system - feel free to steal it to use in your
own projects
Take care,
-AlexOn Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <@Alex_Barry> wrote:
My website just went down, so I remembered my google pages website, and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress. I now
have
the camera system built and have done some testing and I think within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <@Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system, but I’m at
my
parent’s place helping them out this week, so probably closer to the end
of
the week, I’ll have something working, maybe with some configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for their
project, and that adds a lot of bulk to a project that doesn’t need it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done,
i’ll
post
the code to link it up with SDL’s events (probably as a user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes,
the
other
covered from the camera, but pointed at his eye. I put the camera
on
infrared mod, and I should get the light near the middle of his
face,
and
the light reflecting off his eye - from there, I’d assume the
pupil
would
reflect the most, so I just compare the distance/angle between the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. Light reflects off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. You need a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location
on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for
blink
detection, and I could use three lights, one for each eye, and the
last
one
for a reference point.
You should only need one light. Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not
just
wherever you point it. You could recognize the position of his head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes by
the higher reflectivity. That’s before you actually get around to
figuring out where he’s looking though.
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to one.On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <alex.barry at gmail.com> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <alex.barry at gmail.com> wrote:
My website just went down, so I remembered my google pages website, and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress.? I now
have
the camera system built and have done some testing and I think within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <alex.barry at gmail.com> wrote:
Right now I’m working on building the glasses/camera system, but I’m at
my
parent’s place helping them out this week, so probably closer to the
end of
the week, I’ll have something working, maybe with some configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for
their
project, and that adds a lot of bulk to a project that doesn’t need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done,
i’ll
post
the code to link it up with SDL’s events (probably as a user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes,
the
other
covered from the camera, but pointed at his eye. ?I put the
camera
on
infrared mod, and I should get the light near the middle of his
face,
and
the light reflecting off his eye - from there, I’d assume the
pupil
would
reflect the most, so I just compare the distance/angle between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil is
where you would find an additional reflection. ?Light reflects
off
both the outer and inner surface of the lens. ?If you compare the
positions of the reflections, you can find the direction the eye is
focused on relative to the light source and camera. ?You need a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of
the
eye. ?You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a location
on
screen, you could roughly track large eye movements instead and use
them to bump a cursor around. ?So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? ?This would also make it easy for
blink
detection, and I could use three lights, one for each eye, and
the
last
one
for a reference point.
You should only need one light. ?Unless you’re using a laser (which
you shouldn’t), each light will reflect off his entire face, not
just
wherever you point it. ?You could recognize the position of his
head
by contrast with the background (white wall, pillow, etc) (this is
where OpenCV might help), then recognize the position of his eyes
by
the higher reflectivity. ?That’s before you actually get around to
figuring out where he’s looking though.
have
the camera system built and have done some testing and I think within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <@Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system, but I’m
at
my
parent’s place helping them out this week, so probably closer to the
end of
the week, I’ll have something working, maybe with some configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for
their
project, and that adds a lot of bulk to a project that doesn’t need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his eyes,
the
other
covered from the camera, but pointed at his eye. I put the
camera
on
infrared mod, and I should get the light near the middle of his
face,
and
the light reflecting off his eye - from there, I’d assume the
pupil
would
reflect the most, so I just compare the distance/angle between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil
is
where you would find an additional reflection. Light reflects
off
both the outer and inner surface of the lens. If you compare the
positions of the reflections, you can find the direction the eye
is
focused on relative to the light source and camera. You need a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil by
color, since there’s a big contrast between them and the rest of
the
eye. You can use the “red eye” effect to increase this contrast.
If you can’t track the eye accurately enough to pinpoint a
location
on
screen, you could roughly track large eye movements instead and
use
them to bump a cursor around. So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to the
right, etc.
Does this sound reasonable? This would also make it easy for
blink
detection, and I could use three lights, one for each eye, and
the
last
one
for a reference point.
You should only need one light. Unless you’re using a laser
(which
you shouldn’t), each light will reflect off his entire face, not
just
wherever you point it. You could recognize the position of his
head
by contrast with the background (white wall, pillow, etc) (this
is
where OpenCV might help), then recognize the position of his eyes
by
the higher reflectivity. That’s before you actually get around
to
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <alex.barry at gmail.com> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki <@Donny_Viszneki> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <alex.barry at gmail.com> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <alex.barry at gmail.com> wrote:
My website just went down, so I remembered my google pages website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress.? I
now
have
the camera system built and have done some testing and I think within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <alex.barry at gmail.com> wrote:
Right now I’m working on building the glasses/camera system, but I’m
at
my
parent’s place helping them out this week, so probably closer to the
end of
the week, I’ll have something working, maybe with some configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for
their
project, and that adds a lot of bulk to a project that doesn’t need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his
eyes,
the
other
covered from the camera, but pointed at his eye. ?I put the
camera
on
infrared mod, and I should get the light near the middle of
his
face,
and
the light reflecting off his eye - from there, I’d assume the
pupil
would
reflect the most, so I just compare the distance/angle between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the pupil
is
where you would find an additional reflection. ?Light reflects
off
both the outer and inner surface of the lens. ?If you compare
the
positions of the reflections, you can find the direction the eye
is
focused on relative to the light source and camera. ?You need a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil
by
color, since there’s a big contrast between them and the rest of
the
eye. ?You can use the “red eye” effect to increase this
contrast.
If you can’t track the eye accurately enough to pinpoint a
location
on
screen, you could roughly track large eye movements instead and
use
them to bump a cursor around. ?So if he looks left, it bumps the
cursor to the left, if he looks right, it bumps the cursor to
the
right, etc.
Does this sound reasonable? ?This would also make it easy for
blink
detection, and I could use three lights, one for each eye, and
the
last
one
for a reference point.
You should only need one light. ?Unless you’re using a laser
(which
you shouldn’t), each light will reflect off his entire face, not
just
wherever you point it. ?You could recognize the position of his
head
by contrast with the background (white wall, pillow, etc) (this
is
where OpenCV might help), then recognize the position of his
eyes
by
the higher reflectivity. ?That’s before you actually get around
to
figuring out where he’s looking though.
I used a hacked PS3Eye Camera - check the README file for materials - I’ll
write a tutorial for building, but you could always just follow the
instructions from EyeWriter: http://fffff.at/eyewriter/The-EyeWriter.pdf
To have the PS3Eye Camera (remove IR filter, and add an IR bypass): http://nuigroup.com/forums/viewthread/4189/
^ This probably also works for other cameras, but for the quality and price
of a PS3Eye, you’re better off using a PS3Eye (about $40CDN or $35US), where
similar specs on “real” webcams cost about $60-$70…but that’s your call.
If you already have a webcam, google to see if it has an ir filter - if it
does, it should be fairly simple to remove it and put in an ir bypass (check
out the video on hacking the PS3Eye Camera for what to use for an ir bypass)
There is also a website that sells pre-hacked PS3Eye Cameras for this very
purpose, if you’re so inclined : http://peauproductions.com/
I hope that helps,
-AlexOn Thu, Jul 22, 2010 at 2:04 PM, Donny Viszneki <donny.viszneki at gmail.com>wrote:
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.
On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <@Alex_Barry> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki < donny.viszneki at gmail.com> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <@Alex_Barry> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <@Alex_Barry> wrote:
My website just went down, so I remembered my google pages website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress. I
now
have
the camera system built and have done some testing and I think
within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <@Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system, but
I’m
at
my
parent’s place helping them out this week, so probably closer to
the
end of
the week, I’ll have something working, maybe with some
configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up ripping
EyeWriter apart and just using opencv (they use openframeworks for
their
project, and that adds a lot of bulk to a project that doesn’t
need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s
done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull < llubnek at gmail.com> wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his
eyes,
the
other
covered from the camera, but pointed at his eye. I put the
camera
on
infrared mod, and I should get the light near the middle of
his
face,
and
the light reflecting off his eye - from there, I’d assume
the
pupil
would
reflect the most, so I just compare the distance/angle
between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the
pupil
is
where you would find an additional reflection. Light
reflects
off
both the outer and inner surface of the lens. If you compare
the
positions of the reflections, you can find the direction the
eye
is
focused on relative to the light source and camera. You need
a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or pupil
by
color, since there’s a big contrast between them and the rest
of
the
eye. You can use the “red eye” effect to increase this
contrast.
I have previously confirmed that my camera receives infrared light. It
should be as simple then as taping an IR-pass filter to filter out
visible light?On Thu, Jul 22, 2010 at 6:14 PM, Alex Barry <alex.barry at gmail.com> wrote:
I used a hacked PS3Eye Camera - check the README file for materials - I’ll
write a tutorial for building, but you could always just follow the
instructions from EyeWriter: http://fffff.at/eyewriter/The-EyeWriter.pdf
To have the PS3Eye Camera (remove IR filter, and add an IR bypass): http://nuigroup.com/forums/viewthread/4189/
^ This probably also works for other cameras, but for the quality and price
of a PS3Eye, you’re better off using a PS3Eye (about $40CDN or $35US), where
similar specs on “real” webcams cost about $60-$70…but that’s your call.
If you already have a webcam, google to see if it has an ir filter - if it
does, it should be fairly simple to remove it and put in an ir bypass (check
out the video on hacking the PS3Eye Camera for what to use for an ir bypass)
There is also a website that sells pre-hacked PS3Eye Cameras for this very
purpose, if you’re so inclined : http://peauproductions.com/
I hope that helps,
-Alex
On Thu, Jul 22, 2010 at 2:04 PM, Donny Viszneki <@Donny_Viszneki> wrote:
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.
On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <alex.barry at gmail.com> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki <@Donny_Viszneki> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <alex.barry at gmail.com> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <alex.barry at gmail.com> wrote:
My website just went down, so I remembered my google pages
website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress.
I
now
have
the camera system built and have done some testing and I think
within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <alex.barry at gmail.com> wrote:
Right now I’m working on building the glasses/camera system, but
I’m
at
my
parent’s place helping them out this week, so probably closer to
the
end of
the week, I’ll have something working, maybe with some
configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up
ripping
EyeWriter apart and just using opencv (they use openframeworks
for
their
project, and that adds a lot of bulk to a project that doesn’t
need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s
done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his
eyes,
the
other
covered from the camera, but pointed at his eye. ?I put the
camera
on
infrared mod, and I should get the light near the middle of
his
face,
and
the light reflecting off his eye - from there, I’d assume
the
pupil
would
reflect the most, so I just compare the distance/angle
between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the
pupil
is
where you would find an additional reflection. ?Light
reflects
off
both the outer and inner surface of the lens. ?If you compare
the
positions of the reflections, you can find the direction the
eye
is
focused on relative to the light source and camera. ?You need
a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or
pupil
by
color, since there’s a big contrast between them and the rest
of
the
eye. ?You can use the “red eye” effect to increase this
contrast.
If you can’t track the eye accurately enough to pinpoint a
location
on
screen, you could roughly track large eye movements instead
and
use
them to bump a cursor around. ?So if he looks left, it bumps
the
cursor to the left, if he looks right, it bumps the cursor to
the
right, etc.
Does this sound reasonable? ?This would also make it easy
for
blink
detection, and I could use three lights, one for each eye,
and
the
last
one
for a reference point.
You should only need one light. ?Unless you’re using a laser
(which
you shouldn’t), each light will reflect off his entire face,
not
just
wherever you point it. ?You could recognize the position of
his
head
by contrast with the background (white wall, pillow, etc)
(this
is
where OpenCV might help), then recognize the position of his
eyes
by
the higher reflectivity. ?That’s before you actually get
around
to
figuring out where he’s looking though.
That would probably work - my best suggestion is giving it a try, and if
that works, awesome, and if not, the filter might have to be put inside the
lens-system so it’s closer to the camera sensor on the circuit board.
Let me know if that works.
-AlexOn Thu, Jul 22, 2010 at 6:46 PM, Donny Viszneki <donny.viszneki at gmail.com>wrote:
I have previously confirmed that my camera receives infrared light. It
should be as simple then as taping an IR-pass filter to filter out
visible light?
On Thu, Jul 22, 2010 at 6:14 PM, Alex Barry <@Alex_Barry> wrote:
I used a hacked PS3Eye Camera - check the README file for materials -
I’ll
write a tutorial for building, but you could always just follow the
instructions from EyeWriter: http://fffff.at/eyewriter/The-EyeWriter.pdf
To have the PS3Eye Camera (remove IR filter, and add an IR bypass): http://nuigroup.com/forums/viewthread/4189/
^ This probably also works for other cameras, but for the quality and
price
of a PS3Eye, you’re better off using a PS3Eye (about $40CDN or $35US),
where
similar specs on “real” webcams cost about $60-$70…but that’s your
call.
If you already have a webcam, google to see if it has an ir filter - if
it
does, it should be fairly simple to remove it and put in an ir bypass
(check
out the video on hacking the PS3Eye Camera for what to use for an ir
bypass)
There is also a website that sells pre-hacked PS3Eye Cameras for this
very
purpose, if you’re so inclined : http://peauproductions.com/
I hope that helps,
-Alex
On Thu, Jul 22, 2010 at 2:04 PM, Donny Viszneki < donny.viszneki at gmail.com> wrote:
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.
On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <@Alex_Barry> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki <donny.viszneki at gmail.com> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <@Alex_Barry> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <@Alex_Barry wrote:
My website just went down, so I remembered my google pages
website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my progress.
I
now
have
the camera system built and have done some testing and I think
within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry < @Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system,
but
I’m
at
my
parent’s place helping them out this week, so probably closer
to
the
end of
the week, I’ll have something working, maybe with some
configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up
ripping
EyeWriter apart and just using opencv (they use openframeworks
for
their
project, and that adds a lot of bulk to a project that doesn’t
need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga < slouken at libsdl.org> wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s
done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween his
eyes,
the
other
covered from the camera, but pointed at his eye. I put
the
camera
on
infrared mod, and I should get the light near the middle
of
his
face,
and
the light reflecting off his eye - from there, I’d assume
the
pupil
would
reflect the most, so I just compare the distance/angle
between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the
pupil
is
where you would find an additional reflection. Light
reflects
off
both the outer and inner surface of the lens. If you
compare
the
positions of the reflections, you can find the direction
the
eye
is
focused on relative to the light source and camera. You
need
a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or
pupil
by
color, since there’s a big contrast between them and the
rest
of
the
eye. You can use the “red eye” effect to increase this
contrast.
If you can’t track the eye accurately enough to pinpoint a
location
on
screen, you could roughly track large eye movements instead
and
use
them to bump a cursor around. So if he looks left, it
bumps
the
cursor to the left, if he looks right, it bumps the cursor
to
the
right, etc.
Does this sound reasonable? This would also make it easy
for
blink
detection, and I could use three lights, one for each
eye,
and
the
last
one
for a reference point.
You should only need one light. Unless you’re using a
laser
(which
you shouldn’t), each light will reflect off his entire
face,
not
just
wherever you point it. You could recognize the position of
his
head
by contrast with the background (white wall, pillow, etc)
(this
is
where OpenCV might help), then recognize the position of
his
eyes
by
the higher reflectivity. That’s before you actually get
around
to
figuring out where he’s looking though.
Update: I now host my code on google code, and keep source packages (and
maybe some binaries if anyone wants to send me binaries for win32 or mac)
-AlexOn Thu, Jul 22, 2010 at 7:44 PM, Alex Barry <@Alex_Barry> wrote:
That would probably work - my best suggestion is giving it a try, and if
that works, awesome, and if not, the filter might have to be put inside the
lens-system so it’s closer to the camera sensor on the circuit board.
Let me know if that works.
-Alex
On Thu, Jul 22, 2010 at 6:46 PM, Donny Viszneki <donny.viszneki at gmail.com>wrote:
I have previously confirmed that my camera receives infrared light. It
should be as simple then as taping an IR-pass filter to filter out
visible light?
On Thu, Jul 22, 2010 at 6:14 PM, Alex Barry <@Alex_Barry> wrote:
I used a hacked PS3Eye Camera - check the README file for materials -
I’ll
write a tutorial for building, but you could always just follow the
instructions from EyeWriter: http://fffff.at/eyewriter/The-EyeWriter.pdf
To have the PS3Eye Camera (remove IR filter, and add an IR bypass): http://nuigroup.com/forums/viewthread/4189/
^ This probably also works for other cameras, but for the quality and
price
of a PS3Eye, you’re better off using a PS3Eye (about $40CDN or $35US),
where
similar specs on “real” webcams cost about $60-$70…but that’s your
call.
If you already have a webcam, google to see if it has an ir filter - if
it
does, it should be fairly simple to remove it and put in an ir bypass
(check
out the video on hacking the PS3Eye Camera for what to use for an ir
bypass)
There is also a website that sells pre-hacked PS3Eye Cameras for this
very
purpose, if you’re so inclined : http://peauproductions.com/
I hope that helps,
-Alex
On Thu, Jul 22, 2010 at 2:04 PM, Donny Viszneki < donny.viszneki at gmail.com> wrote:
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.
On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <@Alex_Barry> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki <donny.viszneki at gmail.com> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to
one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <@Alex_Barry> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga <slouken at libsdl.org wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry < @Alex_Barry> wrote:
My website just went down, so I remembered my google pages
website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my
progress.
I
now
have
the camera system built and have done some testing and I think
within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry < @Alex_Barry> wrote:
Right now I’m working on building the glasses/camera system,
but
I’m
at
my
parent’s place helping them out this week, so probably closer
to
the
end of
the week, I’ll have something working, maybe with some
configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up
ripping
EyeWriter apart and just using opencv (they use openframeworks
for
their
project, and that adds a lot of bulk to a project that doesn’t
need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga < slouken at libsdl.org> wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <@Alex_Barry> wrote:
After doing a lot of searching, it looks like by best bet
is
something
called “EyeWriter” - it’ll take a few tweaks, but once it’s
done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <@Alex_Barry> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween
his
eyes,
the
other
covered from the camera, but pointed at his eye. I put
the
camera
on
infrared mod, and I should get the light near the middle
of
his
face,
and
the light reflecting off his eye - from there, I’d
assume
the
pupil
would
reflect the most, so I just compare the distance/angle
between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but the
pupil
is
where you would find an additional reflection. Light
reflects
off
both the outer and inner surface of the lens. If you
compare
the
positions of the reflections, you can find the direction
the
eye
is
focused on relative to the light source and camera. You
need
a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or
pupil
by
color, since there’s a big contrast between them and the
rest
of
the
eye. You can use the “red eye” effect to increase this
contrast.
What kinds of games are available and/or popular for people who depend
on this kind of software?On Thu, Jul 22, 2010 at 9:14 PM, Alex Barry <alex.barry at gmail.com> wrote:
Update: I now host my code on google code, and keep source packages (and
maybe some binaries if anyone wants to send me binaries for win32 or mac)
-Alex
On Thu, Jul 22, 2010 at 7:44 PM, Alex Barry <alex.barry at gmail.com> wrote:
That would probably work - my best suggestion is giving it a try, and if
that works, awesome, and if not, the filter might have to be put inside the
lens-system so it’s closer to the camera sensor on the circuit board.
Let me know if that works.
-Alex
On Thu, Jul 22, 2010 at 6:46 PM, Donny Viszneki <@Donny_Viszneki> wrote:
I have previously confirmed that my camera receives infrared light. It
should be as simple then as taping an IR-pass filter to filter out
visible light?
On Thu, Jul 22, 2010 at 6:14 PM, Alex Barry <alex.barry at gmail.com> wrote:
I used a hacked PS3Eye Camera - check the README file for materials -
I’ll
write a tutorial for building, but you could always just follow the
instructions from EyeWriter: http://fffff.at/eyewriter/The-EyeWriter.pdf
To have the PS3Eye Camera (remove IR filter, and add an IR bypass): http://nuigroup.com/forums/viewthread/4189/
^ This probably also works for other cameras, but for the quality and
price
of a PS3Eye, you’re better off using a PS3Eye (about $40CDN or $35US),
where
similar specs on “real” webcams cost about $60-$70…but that’s your
call.
If you already have a webcam, google to see if it has an ir filter - if
it
does, it should be fairly simple to remove it and put in an ir bypass
(check
out the video on hacking the PS3Eye Camera for what to use for an ir
bypass)
There is also a website that sells pre-hacked PS3Eye Cameras for this
very
purpose, if you’re so inclined : http://peauproductions.com/
I hope that helps,
-Alex
On Thu, Jul 22, 2010 at 2:04 PM, Donny Viszneki <@Donny_Viszneki> wrote:
What kind of hardware is needed to develop this software? Google just
showed me an add for a 500 FPS camera used for eye-tracking
applications, and well, that got me wondering if my Microsoft webcam
would do the trick.
On Thu, Jul 22, 2010 at 1:35 PM, Alex Barry <alex.barry at gmail.com> wrote:
heh - it’s on the todo list
On Thu, Jul 22, 2010 at 10:37 AM, Donny Viszneki <@Donny_Viszneki> wrote:
Is there not a revision control repository somewhere? I recommend
against Subversion or CVS if you have not yet committed (lol) to
one.
On Thu, Jul 22, 2010 at 1:14 AM, Alex Barry <alex.barry at gmail.com> wrote:
On Wed, Jul 21, 2010 at 9:42 PM, Sam Lantinga wrote:
Nice!
On Tue, Jul 20, 2010 at 9:29 AM, Alex Barry <alex.barry at gmail.com> wrote:
My website just went down, so I remembered my google pages
website,
and
converted it to my new programming site. http://sites.google.com/site/alexbarry/
Check out the projects page to keep up to date with my
progress.
I
now
have
the camera system built and have done some testing and I think
within
this
week, I’ll have an official release.
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:27 PM, Alex Barry <alex.barry at gmail.com> wrote:
Right now I’m working on building the glasses/camera system,
but
I’m
at
my
parent’s place helping them out this week, so probably closer
to
the
end of
the week, I’ll have something working, maybe with some
configuration
stuff,
too.
Ya, Sam, it’s a pretty neat project - I’ll probably end up
ripping
EyeWriter apart and just using opencv (they use
openframeworks
for
their
project, and that adds a lot of bulk to a project that
doesn’t
need
it).
Take care,
-Alex
On Mon, Jul 5, 2010 at 1:15 PM, Sam Lantinga wrote:
This sounds very cool, I’d love to hear what comes of this!
On Sun, Jul 4, 2010 at 5:02 AM, Alex Barry <alex.barry at gmail.com> wrote:
After doing a lot of searching, it looks like by best bet
is
something
called “EyeWriter” - it’ll take a few tweaks, but once
it’s
done,
i’ll
post
the code to link it up with SDL’s events (probably as a
user-event),
as
well
as a demo application.
Thanks,
-Alex
On Sat, Jul 3, 2010 at 11:05 PM, Kenneth Bull wrote:
On 03/07/2010, Alex Barry <alex.barry at gmail.com> wrote:
Maybe the community can come up with some creative
suggestions?
This is probably how I’ll have to approach it:
Have two infrared lights, one approximately inbetween
his
eyes,
the
other
covered from the camera, but pointed at his eye. ?I put
the
camera
on
infrared mod, and I should get the light near the
middle of
his
face,
and
the light reflecting off his eye - from there, I’d
assume
the
pupil
would
reflect the most, so I just compare the distance/angle
between
the
two
lights to approximate an x/y coordinate pair.
I don’t think the pupil is more reflective really, but
the
pupil
is
where you would find an additional reflection. ?Light
reflects
off
both the outer and inner surface of the lens. ?If you
compare
the
positions of the reflections, you can find the direction
the
eye
is
focused on relative to the light source and camera. ?You
need
a
really
good camera and a dark room though.
Alternatively, you can track the position of the iris or
pupil
by
color, since there’s a big contrast between them and the
rest
of
the
eye. ?You can use the “red eye” effect to increase this
contrast.
If you can’t track the eye accurately enough to pinpoint
a
location
on
screen, you could roughly track large eye movements
instead
and
use
them to bump a cursor around. ?So if he looks left, it
bumps
the
cursor to the left, if he looks right, it bumps the
cursor to
the
right, etc.
Does this sound reasonable? ?This would also make it
easy
for
blink
detection, and I could use three lights, one for each
eye,
and
the
last
one
for a reference point.
You should only need one light. ?Unless you’re using a
laser
(which
you shouldn’t), each light will reflect off his entire
face,
not
just
wherever you point it. ?You could recognize the position
of
his
head
by contrast with the background (white wall, pillow, etc)
(this
is
where OpenCV might help), then recognize the position of
his
eyes
by
the higher reflectivity. ?That’s before you actually get
around
to
figuring out where he’s looking though.
