# `Latu.Column`
[🔗](https://github.com/zero-one-group/latu/blob/v0.4.0/lib/latu/column.ex#L1)

Expressions: column references, literals, operators.

    import Latu.Column

    Latu.filter(df, all([greater(:price, 100), not_equal(:suburb, "Reservoir")]))

Names match `Explorer.Series` wherever the operation is the same one. Latu already depends on
Explorer, so there is no reason to make you carry two vocabularies.

Expressions are values. Extract one into a function, put a list of them through `Enum`, hand
them around — nothing here is a macro.

`asc/1` and friends build a *sort key*, not an expression: Spark's `SortOrder` is its own
message. `asc` puts nulls first and `desc` puts them last, which is SQL's asymmetry and
PySpark's.

This module is PySpark's `Column` *methods* — operators, predicates, casts, sort keys and
`over/2`. Spark's free functions are `Latu.Functions`, which is aliased rather than imported.
Where Spark offers both spellings of one idea (`Column.isNull` and the SQL function `isnull`),
**this module wins and `Latu.Functions` does not wrap the twin** — nine of them; see
`docs/deviations.md`.

There is no `as` here: an expression is named where it is projected — `select(df, total:
F.sum(:price))`, `Latu.agg/2`, `Latu.with_columns/2`, `Latu.observe/3` — so nothing in this
module shares a name with `Latu`, and importing both is clean. `Latu.Plan.as/2` is the bare
alias if a plan needs one.

    def expensive, do: greater(:price, 100)

    Latu.filter(df, all([expensive() | extra]))

# `add`

```elixir
@spec add(term(), term()) :: Latu.Plan.expression()
```

Spark's `+`.

# `all`

```elixir
@spec all([term()]) :: Latu.Plan.expression()
```

Every predicate holds.

Left-folded, so the tree matches PySpark's `a & b & c`. One predicate is itself, and none is
`lit(true)` — the identity, so a filtered list of predicates composes without a branch.

# `any`

```elixir
@spec any([term()]) :: Latu.Plan.expression()
```

Any predicate holds. `all/1`, with `or`; none is `lit(false)`.

# `asc`

```elixir
@spec asc(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.asc`.

# `asc_nulls_first`

```elixir
@spec asc_nulls_first(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.asc_nulls_first`.

# `asc_nulls_last`

```elixir
@spec asc_nulls_last(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.asc_nulls_last`.

# `between`

```elixir
@spec between(term(), term(), term()) :: Latu.Plan.expression()
```

Between two bounds, inclusive.

Not a Spark function: PySpark composes it as `(c >= lower) and (c <= upper)`, and so does
this.

# `cast`

```elixir
@spec cast(term(), String.t()) :: Latu.Plan.expression()
```

Cast to a Spark type, spelled as SQL spells it. See `Latu.Plan.cast/2`.

# `col`

```elixir
@spec col(String.t() | atom()) :: Latu.Plan.expression()
```

A column reference. See `Latu.Plan.col/1`.

# `contains`

```elixir
@spec contains(term(), term()) :: Latu.Plan.expression()
```

Spark's `contains`.

# `desc`

```elixir
@spec desc(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.desc`.

# `desc_nulls_first`

```elixir
@spec desc_nulls_first(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.desc_nulls_first`.

# `desc_nulls_last`

```elixir
@spec desc_nulls_last(term()) :: Latu.Plan.sort_order()
```

A sort key, as PySpark's `Column.desc_nulls_last`.

# `divide`

```elixir
@spec divide(term(), term()) :: Latu.Plan.expression()
```

Spark's `/`.

# `ends_with`

```elixir
@spec ends_with(term(), term()) :: Latu.Plan.expression()
```

Spark's `endsWith`.

# `equal`

```elixir
@spec equal(term(), term()) :: Latu.Plan.expression()
```

Spark's `==`.

# `equal_null_safe`

```elixir
@spec equal_null_safe(term(), term()) :: Latu.Plan.expression()
```

Spark's `<=>`.

# `expr`

```elixir
@spec expr(String.t()) :: Latu.Plan.expression()
```

Raw SQL, parsed by the server. See `Latu.Plan.expr/1`.

# `fun`

```elixir
@spec fun(String.t(), [term()], keyword()) :: Latu.Plan.expression()
```

Any Spark function, by name.

    fun("upper", [:suburb])

The escape hatch for a function `Latu.Functions` has no wrapper for. See `Latu.Plan.fun/3`;
`distinct: true` is the one option.

# `greater`

```elixir
@spec greater(term(), term()) :: Latu.Plan.expression()
```

Spark's `>`.

# `greater_equal`

```elixir
@spec greater_equal(term(), term()) :: Latu.Plan.expression()
```

Spark's `>=`.

# `ilike`

```elixir
@spec ilike(term(), term()) :: Latu.Plan.expression()
```

Spark's `ilike`.

# `ilike`

```elixir
@spec ilike(term(), term(), term()) :: Latu.Plan.expression()
```

Case-insensitive `like/3`.

# `is_nan`

```elixir
@spec is_nan(term()) :: Latu.Plan.expression()
```

Spark's `isNaN`.

# `is_not_null`

```elixir
@spec is_not_null(term()) :: Latu.Plan.expression()
```

Spark's `isNotNull`.

# `is_null`

```elixir
@spec is_null(term()) :: Latu.Plan.expression()
```

Spark's `isNull`.

# `isin`

```elixir
@spec isin(term(), term()) :: Latu.Plan.expression()
```

One of these values, or one of a DataFrame's rows.

    isin(:suburb, ["Reservoir", "Northcote"])
    isin(:id, Latu.select(recent, :id))

Spark calls the function `in`, which Elixir cannot, so this keeps PySpark's name.

A DataFrame builds an IN subquery rather than a function call, as `Column.isin` does in
PySpark: the frame is hoisted into the plan that uses it and has to have one column — or one
per field, when the left side is a `F.struct/1`. No same-session check, as in
`Latu.DataFrame.scalar/1`.

# `less`

```elixir
@spec less(term(), term()) :: Latu.Plan.expression()
```

Spark's `<`.

# `less_equal`

```elixir
@spec less_equal(term(), term()) :: Latu.Plan.expression()
```

Spark's `<=`.

# `like`

```elixir
@spec like(term(), term()) :: Latu.Plan.expression()
```

Spark's `like`.

# `like`

```elixir
@spec like(term(), term(), term()) :: Latu.Plan.expression()
```

A SQL `LIKE` pattern with an explicit escape character.

`like/2` sends two arguments and this sends three — Spark distinguishes them, so this is a
second clause rather than a default. `Latu.Functions` deliberately does not also wrap `like`:
the Column spelling owns it. See `docs/deviations.md`.

# `lit`

```elixir
@spec lit(term()) :: Latu.Plan.expression()
```

A typed literal. See `Latu.Plan.lit/1`.

# `multiply`

```elixir
@spec multiply(term(), term()) :: Latu.Plan.expression()
```

Spark's `*`.

# `not_`

```elixir
@spec not_(term()) :: Latu.Plan.expression()
```

Negate a predicate. `not` is an operator — `def not(x)` is a syntax error.

# `not_equal`

```elixir
@spec not_equal(term(), term()) :: Latu.Plan.expression()
```

Not equal.

Spark has no `!=` function: PySpark negates `==`, and the `op_compare` fixture is why this is
not a row in the table above.

# `over`

```elixir
@spec over(Latu.Plan.expression(), Latu.Window.t()) :: Latu.Plan.expression()
```

Evaluate an expression over a window.

    alias Latu.Window, as: W

    window = W.partition_by([:suburb]) |> W.order_by([desc(:price)])

    Latu.with_columns(df, rank: over(F.rank(), window))

A window with no `partition_by` moves every row into one partition. Spark allows it and it is
occasionally what you want, so this warns rather than refusing — as PySpark does.
`W.partition_by([])` says the global window is meant, and is not warned about.

# `pow`

```elixir
@spec pow(term(), term()) :: Latu.Plan.expression()
```

Spark's `power`.

# `remainder`

```elixir
@spec remainder(term(), term()) :: Latu.Plan.expression()
```

Spark's `%`.

# `rlike`

```elixir
@spec rlike(term(), term()) :: Latu.Plan.expression()
```

Spark's `rlike`.

# `star`

```elixir
@spec star() :: Latu.Plan.expression()
```

Every column.

# `starts_with`

```elixir
@spec starts_with(term(), term()) :: Latu.Plan.expression()
```

Spark's `startsWith`.

# `subtract`

```elixir
@spec subtract(term(), term()) :: Latu.Plan.expression()
```

Spark's `-`.

# `try_cast`

```elixir
@spec try_cast(term(), String.t()) :: Latu.Plan.expression()
```

`cast/2`, but null where a cast would fail. See `Latu.Plan.try_cast/2`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*