First, let’s start with definition of Closure, as found in Wikipedia:
In programming languages, a closure (also lexical closure or function
closure) is a function or reference to a function together with a
referencing environment—a table storing a reference to each of the
non-local variables (also called free variables or upvalues) of that
function.
Closure is the term that is used to refer to a function along with the variables from its environment that it “closes”.
The definition of Closure in Swift is inline with lambdas and blocks in other languages like C# and Ruby.
As for the difference from functions, from the Swift documentation:
Global and nested functions, as introduced in Functions, are actually
special cases of closures
So all functions are essentially closures that store references to variables in their context.
Closure expressions are convenient way of writing closures, and provides more terse syntax.