robobait

Archived Posts from this Category

FFmpeg’s fps filter, documented

Posted by on 30 Apr 2020 | Tagged as: FFmpeg, robobait, software engineering, web technology

The FFmpeg media editing software is a valuable tool, but its documentation is only barely adequate. It certainly does not answer all the questions I have, as a user trying to understand why FFmpeg is not doing what I want it to do.

Fortunately, FFmpeg is open source, so when the documentation fails, one can read the source. I wanted to learn about presentation time stamps and time bases. The fps video filter source code, in file libavfilter/vf_fps.c, was an instructive read.

I took what I learned from reading that source, and did a complete rewrite of the fps filter documentation. It is longer than the original fps filter documentation (as archived in April 2020 — you can check if the present documentation is any better). I believe the rewrite is more complete and more accurate. I contributed the rewrite to the FFmpeg project. I submitted it as a patch to the ffmpeg developers list. Discussion continues. I don’t know if this contribution will ultimately get accepted.

So, for the benefit of FFmpeg users who are web-searching for answers, here is my documentation of FFmpeg’s fps video filter.


fps

Make a new video from the frames and presentation time stamps (PTSs) of the input. The new video has a specified constant frame rate, and new PTSs. It generally keeps frames from the old video, but might repeat or drop some frames. You can choose the method for rounding from input PTS to output PTS. This affects which frames fps keeps, repeats, or drops.

It accepts the following parameters:

fps

The output frame rate, in frames per second. May be an integer, real, or rational number, or an abbreviation. The default is 25.

start_time

A time, in seconds from the start of the input stream, which fps converts to an input starting PTS and an output starting PTS. If set, fps drops input frames which have PTSs less than the input starting PTS. If not set, the input and output starting PTSs are zero, but fps drops no input frames based on PTS. (See details below.)

round

Rounding method to use when calculating output PTSs from input PTSs. If the calculated output PTS is not exactly an integer, then the value determines which neighbouring integer value fps selects.

Possible values are:

  • zero
    • round towards 0
  • inf
    • round away from 0
  • down
    • round towards -infinity
  • up
    • round towards +infinity
  • near
    • round to nearest (midpoints round away from 0)

The default is near.

eof_action

Action which fps takes with the final input frame. The input video passes in a final input PTS, which fps converts to an output PTS limit. fps drops any input frames with a PTS at or after this limit.

Possible values are:

  • round
    • Use same rounding method as for other frames.
  • pass
    • Round the ending input PTS using up. This might make fps include one last input frame. 

The default is round.

Alternatively, the options may be specified as a flat string:  fps[:start_time[:round]].

fps makes an output video with consecutive integer PTSs, and with a time base set to the inverse of the given frame rate. fps keeps, repeats, or drops input frames, in sequence, to the output video. It does so according to their input PTSs, as converted to seconds (via the input time base), then rounded to output PTSs. 

fps sets output PTSs in terms of a timeline which starts at zero. For any output frame, the integer PTS multiplied by the time base gives a value in seconds on that timeline. If the start_time parameter is not set, or is zero, the first output frame’s PTS is zero. Otherwise, the first PTS is the output starting PTS calculated from the start_time parameter. 

fps interprets input PTSs in terms of the same timeline. It multiplies each input frame’s PTS by the input time base, to get a value in seconds on the timeline. It rounds that value to an integer output PTS. For example, if the input video has a frame rate of 30 fps, a time base of 1/30 seconds, and a first frame with a PTS of 300, then fps treats that first frame as occurring 10 seconds (= 300 * 1/30) after the start of the video, even though it is the first frame.

Setting a start_time value allows for padding/trimming at the start of the input. For example, you can set start_time to 0, to pad the beginning with repeats of the first frame if a video stream starts after the audio stream, or to trim any frames with a negative PTS. When start_time is not set, the fps filter does not pad or trim starting frames, as long as they contain PTSs.

See also the setpts and settb filters.

Details

fps outputs exactly one frame for each output PTS. If there is exactly one input frame with an input PTS which converts to the current output PTS, fps keeps (outputs) that frame. If there are multiple frames which convert to the same output PTS, fps outputs the final frame of that group, and drops the previous frames. 

If the input frame PTS converts to an output PTS later than the current output PTS, fps repeats the previously output frame as the current frame. When this happens for the first input frame,  fps “pads” — outputs repetitions of — that first frame until the output PTS reaches the value converted from that first frame’s input PTS. 

fps always drops input frames which have no PTS set, regardless of the start_time parameter. 

The frame rate value must be zero or greater. It may be provided in a variety of forms. Each form is converted into a rational number, with an integer numerator and denominator. 

  • An integer number, e.g. 25. This converts to the rational number 25/1.
  • A real number, e.g. 3.14145926. This converts to a rational number, e.g. 954708/303893
  • A rational number. The numerator and denominator may be either integers or real numbers. e.g. 30/1.001 or -30000/-1001, which both convert to 30000/1001. The denominator must be non-zero.
  • An abbreviation. e.g ntsc as 30000/1001ntsc-film as 24000/1001. See the complete list at the “Video rate” section in the ffmpeg-utils(1) manual.

fps defines a sync point on the timeline, where one input PTS and one output PTS occur at the same moment. It calculates other PTSs as time offsets from this sync point. This affects the details of rounding. If start_time is set, then fps uses it to calculate input and output PTSs, and makes them the sync point. Otherwise, input and output PTS of zero are the sync point.

Note that fps does not assume that input frames are separated by exactly 1/frame_rate seconds. It takes the input PTSs literally. If the increment of PTS between frames varies along the video, fps treats those frames as happening at varying time intervals. 

An input video with PTSs starting past zero might yield unexpected results. Suppose the input PTSs start at 300, and say that converts to 10 seconds. Then fps repeats the first frame to fill the first 10 seconds of the output video. (However, ffmpeg may suppress those repeated frames, depending on the -vsync setting.) If you set start_time to 10 seconds, then fps sets the sync point to the PTSs converted from 10 seconds on the timeline. It no longer repeats the first frame. And, it starts the output PTS at a value corresponding to 10 seconds, instead of zero.

Examples

  • A typical usage to make a video with a frame rate of 25 frames per second, from an input video with any frame rate:
    • fps=fps=25
  • The output frames have PTSs of 0, 1, 2, etc. The frame rate is 25/1. The time base is 1/25.
  • Make a video with a frame rate of 24 frames per second, using an abbreviation, and rounding method to round to nearest:
    • fps=fps=film:round=near
  • Clean up a video with varying time between frames, and dropped frames. The input video is supposed to have an NTSC standard frame rate of 29.97 frames per second, and the time base is 3003/90000, but the PTSs increment variably at slightly more and less than that rate. The recorder dropped some frames, but the PTSs still reflect when the remaining frames were captured. 
    • fps=fps=30/1.001:round=near
  • The output PTSs are 0, 1, 2, etc. The time between frames is exact. The output frame rate is 30000/1001. The time base is 1001/30000. Where frames were dropped by the recorder, fps repeated frames to fill the gaps.

Happy new decade, yes, 2020 not 2021

Posted by on 31 Jan 2020 | Tagged as: culture, robobait, time

Happy new decade, everyone! This month, January 2020, marks the start of a new decade, the 2020’s. And for those of you who are saying, “no, strictly speaking the decade doesn’t begin until 2021”, you are wrong. Or, at least, all of us who say that decades and centuries start on round numbers are just as “strictly speaking” right as you. Here is why.

I think we all agree that a “decade” generally means a 10-year time span, and a “century” means a 100-year time span. In everyday usage, the word “decade” means a time interval which starts on a year ending in “0”, and ends on a year ending in “9”. Thus we have just finished the “2010’s” (10 years from the start of 2010 to the end of 2019), and are entering the “2020’s” (2020–2029). By the same token, every-day use of the time unit “century” means a time interval which starts on a year ending in “00” and ends on a year ending in “99”. The previous century is known as the “1900’s”, and also as the “20th century”.

And all of the above only applies to English usage of North America, when using the current standard Gregorian Calendar. It is also limited to year 1 onwards — the Common Era (earlier labelled as Anno Domini or A.D. by Bede back in the 600’s and 700’s).
The pedants will point to the first year of the Common Era, which is year 1 rather than year 0. They argue that the first decade must begin with year 1 and be 10 years long, so it must end on year 10. Count forward in 10-year increments, and the 2020’s must start with the year 2021 and end with the year 2030. Similarly, the pedants argue that the first century must begin with year 1 and be 100 years long, so it must end on year 100. Note that the entire foundation of their pedantry is that the time unit “decade” must always and only be 10 years long, and “century” must always and only be 100 years long.

I believe that where the pedants go astray is to prize a constant length over a convenient starting point for “decade” and “century”.

Decades, and calendars, are social constructs. They don’t have to abide by strict mathematics, and they don’t. There is a value to having a calendar which matches the earth’s rotation around the sun. There is value to having a calendar which adjusts to seasonal changes in sunrise and sunset. And, there is value to having a calendar which matches common and convenient language usage.

Consider the day: normally it is 24 hours long. But when “springing forward” from Standard to Daylight Saving Time, there is a day which is 23 hours long. Later in the year, when “falling back” to Standard Time, there is a day which is 25 hours long. I have had to write software which handled this variation. A day is normally 24 hours, but can be 23 or 25 hours.

Consider the minute: normally it is 60 seconds long. But when a leap second is necessary — to keep the highly-accurate Coordinated Universal Time (UTC) in sync with the Earth’s variable orbital duration, and to keep the March equinox close to March 20 — the minute containing the leap second is 61 seconds long.

Some years are 365 days long, others are 366 days long, because of leap years.

Britain officially started its civil year on 25 March, until as late as 1751. Only in 1752 did it finally change the start of the civil year to 1 January. Common usage was ahead of civil rules in adopting 1 January as the beginning of the year, leading dates from 1 January to 24 March to be written with two alternative years: “30 January 1648/1649”.  Even more interestingly, September 1752 was 19 days long in England, instead of 30 days as in other years. This was to bring the calendar back in sync with the solar year, or in other words, to move March 20 back to the day of the March equinox.

All these calendar shenanigans are about deciding to allow social (and astronomical) considerations take precedence over strictly consistent numerical duration. “Strictly speaking”, a calendar balances all these considerations. And rightly so.

Given that, I think it is perfectly reasonable to declare: decades are normally 10 years long, but the first decade of the Common Era was 9 years long: 1 to 9 CE. 10 CE began the second decade. Centuries are normally 100 years long, but the first century CD was 99 years long: 1 to 99 CE.

Thus, 1 Jan 2020 CE also began a decade. Pedants, I see where you are coming from, but if you are willing to use a calendar with 25-hour days, and a 19-day September back in the day, I claim you cannot justify denying the 9-year decade and the 99-year century.

I am tempted to find a time machine, go back to northeast England in the 7th or 8th centuries CE, and persuade Bede to define his Anno Domini year numbers at 0, instead of 1. It would sidestep this whole decade and century problem. The only problem is, that in Bede’s time, Europe would not learn of the zero for another 3-5 centuries (where “centuries” are not strict 100-year time durations).

Happy new decade, starting January 2020!

Duplicate entry names in a single directory on a file server: solved!

Posted by on 31 Aug 2019 | Tagged as: robobait, software engineering

I have just seen — and solved — the most remarkable thing in a deep corner of my large archive disk: a single directory containing two entries (subdirectories) with the same name and same inode number. I will describe the problem, the diagnosis, and the cure for the benefit of others who encounter the same problem.

I was moving my archive of old files from one Network-Attached Storage (NAS) file server on my home network to another. Both old and new servers use netatalk AFP software to present Mac=style volumes to my Mac computer. Both run an underlying Unix-like OSs and file systems (but different ones for each).

I moved the archive by dragging the top-level directory data folder, using Finder on my Mac, from the old server to the new. Partway through, the copy aborted, with an error message like,  “a directory with the name .externalToolBuilders already exists”. This is remarkable. Each directory on the old server might have many entries or few, but each entry must has a different name. It is one of the fundamental rules of file systems. I was not combining two directories together, where an entry from one directory might collide with an entry with the same name from the other directory. Continue Reading »

How to convert Google Docs to Markdown format

Posted by on 30 Apr 2019 | Tagged as: robobait, software engineering

Recently I needed to convert a Google Docs wordprocessing document to Markdown format (Github’s dialect). A simple web search turned up several hits, most of them unhelpful. I finally found a Google Apps script to do the conversion, which was almost, but not quite, suitable. But with a simple modification, it did the trick. I am sharing it here, in the hope that it will be helpful to someone else searching for “convert Google Docs to Markdown”.

Continue Reading »

Top Posts: How to escape apostrophe (‘) in MySql?

Posted by on 28 Feb 2019 | Tagged as: robobait, software engineering, web technology

I post on various forums around the net, and a few of my posts there get some very gratifying kudos. I’ve been a diligent contributor to StackOverflow, the Q-and-A site for software developers. I’m in the top 5% of contributors overall. Here’s my top-voted answer in StackOverflow currently.

The question, How to escape apostrophe (‘) in MySql?,  was asked by anonymous user4951 in March 2012 (and copy-edited by someone else). In abbreviated form, it was:

Continue Reading »

“2 1+ 1 sections”: a quick way to refer to a part of a picture

Posted by on 31 Oct 2018 | Tagged as: robobait, software engineering

For one of my consulting clients, I found myself writing command-line tools that operate on videos. One tool zoomed in on the portion of the video frame, to let the user examine it closely. How do you tell a command-line tool to zoom in on one portion of video frame? I came up with an idea, which I call “2 1+ 1 sections”. It is a quick way for a user to refer to a part of a picture, using a concise text notation. I haven’t used it for that client, but I’ll post it here in case it comes in useful later on. Continue Reading »

Top Posts: Why Unicode has separate codepoints for “characters with identical glyphs”

Posted by on 31 May 2018 | Tagged as: i18n, multilingual, robobait, software engineering, Unicode

I post on various forums around the net. Sometimes I am able to tap into such inspiration that I want to add that essay to my portfolio. Such was the case here. The question: Why does Unicode have separate codepoints for characters with identical glyphs? My response begins: The short answer to this question is, “Unicode encodes characters, not glyphs”. But like many questions about Unicode, a related answer is “plain text may be plain, but it’s not simple”.… Continue Reading »

How to add an SSL certificate to LiClipse to permit EGit access to a git repo

Posted by on 26 Dec 2017 | Tagged as: robobait, software engineering, web technology

I was contributing to the FFmpeg project recently. They keep their source code in a Git repo, accessed via SSL. I had an awkward error message:

SSL reported: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target

The problem was that my tool handling the SSL communication lacked the SSL certificate which validated the communication with the project. I could dismiss the error and proceed without validating the SSL security. The better solution was to supply the right SSL certificate to the communication tool, so that it could validate the SSL security with no awkwardness. Here’s how I accomplished that.  This post is offered as search engine fodder, in hopes that others will benefit from these instructions. Continue Reading »

LiClipse (for Mac) includes its own copy of the JRE

Posted by on 10 Dec 2017 | Tagged as: robobait, software engineering

LiClipse is the developers tool I use for writing Python code. Based on the Eclipse IDE, it accepts numerous plugins to support other programming languages like Java and C, and related tools, such as the Git version control system. Eclipse is mostly Java language code, and it runs on a JRE (Java Runtime Environment). Last month, I wanted to contribute code to a git repository which I accessed via HTTPS. That worked more smoothly if I could put an SSL certificate into the JRE, and I’ll skip the details of why for now.

So I looked up the Java Home of the JRE installed on my Mac OS X laptop (short answer: it’s the path output by running /usr/libexec/java_home). I installed the SSL certificate there. It did not work. That was a sign that LiClipse did not use that JRE. Did it perhaps include its own JRE?  After some investigation, I found out the answer: yes!

Here’s the explanation. I hope this helps others. Continue Reading »

When I run “ffmpeg” in the background, how do I prevent “suspended (tty output)”?

Posted by on 04 Nov 2017 | Tagged as: robobait, software engineering

I recently had a problem, “When I run ffmpeg in the background, how do I prevent suspended (tty output)?”. I solved it. Here is my solution, in the hopes that it will help others seeing the same problem.

I have a sh script which calls ffmpeg on several files. When I try to run this script in the background, redirecting output to a file, the job starts but then immediately suspends:

% bin/mp3convert.sh path/a/b &> ~/tmp/log.txt &
[1] 93352
% [1]  + suspended (tty output)  bin/mp3convert.sh path/a/b &>

Continue Reading »

Next Page »