The compositional data are proportionals of mutually exclusive groups that would be summed up to the unity. Statistical models for compositional data have been applicable in a number of areas,...continue reading.
(require ‘[huri.core :as h] ‘[clojure.core.matrix.dataset :as d] ‘[incanter.core :as i]) ;; FROM MAP OF ROWS TO MAP OF COLUMNS (def byRow [{:x 1 :y “a”} {:x 2 :y “b”} {:x...continue reading.
(require ‘[tnoda.rashinban :as rr] ‘[tnoda.rashinban.core :as rc] ‘[clojure.core.matrix.dataset :as dt] ‘[clojure.core.matrix.impl.dataset :as id]) ;; CREATE A TOY DATA (def ds [{:id 1.0 :name “name1”} {:id 2.0 :n…continue reading.
(require ‘[ultra-csv.core :refer [read-csv]] ‘[criterium.core :refer [quick-bench]] ‘[clojure.set :refer [index]]) (def ds (read-csv “/home/liuwensui/Downloads/nycflights.csv”)) ;; FASTEST (quick-bench (map (fn [x] {:year (f…continue reading.
(require ‘[clojure.pprint :refer [print-table] :rename {print-table p}] ‘[clojure.set :as s] ‘[clojure.core.reducers :as r]) ;; CREATE TOY DATASETS (def ds1 [{:id 1 :name “name1”} {:id 2 :name “name2”} …continue reading.
In the previous post (https://statcompute.wordpress.com/2018/03/16/for-loop-and-map-in-clojure), I did a performance comparison between MAP and FOR loop in Clojure with a small dataset. It is interesting to see that the performance of...continue reading.
During my learning of Clojure, I was deeply impressed by the richness of data structures and the built-in support for concurrency, which makes it a perfect programming language for the...continue reading.
; (defproject prj “0.1.0-SNAPSHOT” ; :dependencies [[org.clojure/clojure “1.8.0”] ; [org.clojars.bmabey/csvlib “0.3.6”] ; [ultra-csv “0.2.1”] ; [incanter “1.5.7”] ; [semantic-csv “0.2.1-alpha1”] ; [org.clojure/data.csv “0.1.4”]]) (require ‘[csvlib :…continue reading.
;; DEFINE THE DATABASE (def db {:classname “org.sqlite.JDBC” :subprotocol “sqlite” :subname “/home/liuwensui/Downloads/chinook.db”}) ;; CALL PACKAGES (require ‘[clojure.java.jdbc :as j] ‘[criterium.core :as c] ‘[cloju…continue reading.
When I learned and experimented a new model, I always like to start with its likelihood function in order to gain a better understanding about the statistical nature. That’s why...continue reading.
After switching the role from the credit risk to the operational risk in 2015, I spent countless weekend hours in the Starbucks researching on how to model operational losses in...continue reading.
Keras is a popular Python package to do the prototyping for deep neural networks with multiple backends, including TensorFlow, CNTK, and Theano. Currently, there are two R interfaces that allow...continue reading.
In my previous post (https://statcompute.wordpress.com/2018/01/28/modeling-lgd-with-proportional-odds-model), I’ve discussed how to use Proportional Odds Models in the LGD model development. In particular, I specifically mentioned that we would estimate a sub-model, which...continue reading.
Similar to COM-Poisson, Double-Poisson, and Generalized Poisson distributions discussed in my previous post (https://statcompute.wordpress.com/2016/11/27/more-about-flexible-frequency-models/), the Hyper-Poisson distribution is another extension of the standard Poisson and is able to accommodate both...continue reading.
The LGD model is an important component in the expected loss calculation. In https://statcompute.wordpress.com/2015/11/01/quasi-binomial-model-in-sas, I discussed how to model LGD with the quasi-binomial regression that is simple and makes no...continue reading.
While the monotonic binning algorithm has been widely used in scorecard and PD model (Probability of Default) developments, the similar idea can be generalized to LGD (Loss Given Default) models....continue reading.
In the post (https://statcompute.wordpress.com/2017/06/15/finer-monotonic-binning-based-on-isotonic-regression), it is shown how to do a finer monotonic binning with isotonic regression in R. Below is a SAS macro implementing the monotonic binning with the...continue reading.
As mentioned in the previous post (https://statcompute.wordpress.com/2017/06/29/model-operational-loss-directly-with-tweedie-glm/), we often need to model non-negative numeric outcomes with zeros in the operational loss model development. Tweedie GLM provides a convenient interface to...continue reading.
LASSO has been a popular algorithm for the variable selection and extremely effective with high-dimension data. However, it often tends to “over-regularize” a model that might be overly compact and...continue reading.
The dropout approach developed by Hinton has been widely employed in deep learnings to prevent the deep neural network from overfitting, as shown in https://statcompute.wordpress.com/2017/01/02/dropout-regularization-in-deep-neural-networks. In the paper http://proceedings.mlr.press/v38/korlakaivinayak15.pdf, the...continue reading.
Artificial intelligence / Deep neural networks / Keras / R / R News / Spark
April 22, 2024
GIS / R / R News / Shiny / Visualization
Interactive visualization of geospatial data with R Shiny
April 17, 2021
