Description
Further, read the paper “The world of interactive
media systems and applications” and consider the fusion of rendering with computer vision and the
concept of rendering a scene that is first fully parsed by computer vision. Do you believe it would ever
be possible for computer vision to fully parse and render a scene so that it is indistinguishable from
reality? – a Turing test for computer vision.
Exercise #3 Requirements:
1) [5 points] Read the paper “The world of interactive media systems and applications“ and
summarize key points of the paper. At one point early in the days of ray tracing and
computer vision, researchers worried about indistinguishable real video from generated
(rendered) video – do you think this was a valid concern?
2) [10 points] Using a single frame from the Laser in a Dark Room, Uncluttered, found here at
Dark-Room-Laser-Spot.mpeg, apply the Median filter as documented in our Lecture-Week-5
to the G band only in a graymap and provide an image of before and after filter images in
your report – did this help enhance the laser spot edge boundary at all? [Use ffmpeg or
avconv to decode and save a single frame from the MPEG video]
3) [15 points] Modify the video, Dark-Room-Laser-Spot-with-Clutter.mpeg and use frame
differencing for R,G & B to remove the bookshelf background and to preserve the moving
laser spot foreground. Re-encode the difference frames that result. How effective was this at
removing clutter? Upload the video with no background with your report.
4) [15 points] Use the Dark Room Laser Spot video and convert the entire video to a grayscale
graymap by using the G band (from RGB) only – re-encode the video. Note that frames
should be saved as a PGM (see http://en.wikipedia.org/wiki/Netpbm_format ) and then reencoded into MPEG4 using ffmpeg. You can use OpenCV if your wish or your own code to
write out the PGM frames.
5) [15 points] Use the Dark Room Laser Spot video you converted to a series of graymap
frames (using G band) and write a threshold function based on your analysis of the
characteristics of the edges of this spot in the G band – raster each frame to determine the Xbar, Y-bar object COM based on X row maximum extents and Y column maximum extents
and your threshold detector. Use the Median Filter and/or Sharpen PSF if you want to
enhance the image prior to threshold detection of the X and Y edges. Mark the COM (Center
of Mass) and track it in each frame with cross-hairs (at saturation level of 255 – should
appear as white lines) so that the re-encoded video has tracking overlay.
6) [20 points] Repeat above exercise, but modify your threshold for RGB space, use the LightRoom-Laser-Spot-with-Clutter.mpeg, use background elimination first, then apply your
COM detector and try to track the spot with an RGB threshold function – re-encode your
graphically annotated video. Apply Sharpen PFS and/or the Median filter if you believe this
will help with reliable edge detection in RGB.
Upload all video as encoded MPEG-4 at a reasonable bit-rate and quality.
[20 points] Overall, provide a well-documented professional report of your findings, output, and
tests so that it is easy for a colleague (or instructor) to understand what you’ve done. Include any
C/C++ source code you write (or modify) and Makefiles needed to build your code and make
sure your code is well commented, documented and follows coding style guidelines. I will look
at your report first, so it must be well written and clearly address each problem providing clear
and concise responses to receive credit.
In this class, you’ll be expected to consult the Linux and OpenCV manual pages and to do some
reading and research on your own, so practice this in this first lab and try to answer as many of
your own questions as possible, but do come to office hours and ask for help if you get stuck.
Upload all code and your report completed using MS Word or as a PDF to Canvas and include
all source code (ideally example output should be integrated into the report directly, but if not,
clearly label in the report and by filename if test and example output is not pasted directly into
the report). Your code must include a Makefile so I can build your solution on an embedded
Linux system (R-Pi 3b+ or Jetson). Please zip or tar.gz your solution with your first and last
name embedded in the directory name and/or provide a GitHub public or private repository
link. Note that I may ask you or SA graders may ask you to walk-through and explain your
code. Any code that you present as your own that is “re-used” and not cited with the original
source is plagiarism. So, be sure to cite code you did not author and be sure you can explain it
in good detail if you do re-use, you must provide a proper citation and prove that you
understand the code you are using.
Grading Rubric
[5 points] Read and summarize the main points…
[1 pts] main point #1 ______________________________________________________
[1 pts] main point #2 ______________________________________________________
[1 pts] main point #3 ______________________________________________________
[2 pts] summary overall ____________________________________________________
[10 points] Dark room laser spot median filter G-band transformation and analysis
[6 pts] Application of median filter to video_ __________________________________
[4 pts] Analysis of helpfulness of median filter transform to enhance edge boundaries
______________________________________________________________________
[15 points] Clutter removal from Dark room laser spot with clutter video
[10 pts] R, G, and B background elimination and code to do so well
commented____________________________________________________________
[5 pts] Comments on effectiveness for removing clutter and example video re-encoded
with background elimination______________________________________________
[15 points] Conversion of Dark room laser spot video to grayscale (G band only)
[10 pts] Conversion from RGB to G only grayscale, code to do so, and PGM frames
(sampling)
_____________________________________________________________________
[5 pts] Re-encoded grayscale only video using ffmepg
(avconv)______________________________________________________________
[15 points] Center of laser spot tracking with enhancement as needed to optimize performance
[5 pts] Code and build, test, run to show tracking in grayscale
_______________________________________________________________________
[10 pts] Analysis of tracking precision and performance along with steps taken to
improve using image
enhancement_____________________________________________________________
[20 points] Center of laser spot tracking with background elimination in RGB space
[10 pts] Code updates and build, test, run to show tracking in
RGB__________________________________________________________________
[10 pts] Video encoding to show graphic overlay of center of laser spot so tracking
precision can be seen in
video__________________________________________________________________
[20 points] Quality of reporting and code quality and originality:
[10 pts] Professional quality of reporting, testing and analysis (0…6 is below average, 7
is average, 8 is good, 9 excellent, and 10 is best
overall.)_________________________________________________________________
[10 pts] Code quality including style, commenting, originality, proper citation for re-used
code, modified code, etc.___________________________________________________
